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lution video signal reconxc r^eon and a system tor 
reproducing the optical Ov* •*rr^ a conventional system 
tr> producing a standard *+%a*jbcn video signal. A high 
resolution video Srgna r* o*x3ec by video division 
means into a mam sipna* j vjfc signal, and the main 
signal and the sut? s*y>-; *<* MPEG-encoded. The 
stream ot the man s>grw: s*»eam of the sub sig- 

nal a»e divided into i GC*-' »ur c* fiames. First inter- 
leave blocks 54 eacfi »uo>v ' o» more ot the 
stream of the mam s*g^* and «*cond interleave blocks 
55 each including l OCX :> more o* me stream of the 
sub signal are recorded cm ar*. optical disk 1 A high res- 
olution reproduction apparatus » cp»oduces both the first 
and second interleave boot obtain a high resolution 
video output. A non-hign o^afat^ picture reproduction 
apparatus reproduces onj> tmst or second interleave 
blocks to obtain a stanoarr resolution video output. 
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Description 
TECHNICAL FIELD 

[0001] The piesenl invention i elates to an optical 
disk having high quality picture or standaid pictuie 
recorded thereon, and a lecording and lepioduciion 
apparatus for such an optical disk 

BACKGROUND ART 

{0002] Foi an optical disk having high quality pictui e 
recorded thereon and a reproduction apparatus tor such 
an optical disk, systems referred to as 480P and 720P 
tor recording piogiessive data have conventionally been 
studied. A conventionally known reproduction control 
system lor an optical disk uses one MPEG decoder. 
{0003} First, a first problem ol the conventional sys- 
tem will be describee. When a conventional optical disk 
tor high quality picture recording is reproduced by a 
standard reproduction apparatus, normal pictures can- 
not be output. The optical disk for high quality picture 
recording can only be reproduced by a high quality pic- 
ture reproduction apparatus. Accordingly, there is a 
need to produce two types o1 optical disks having the 
same contents. In other words, the conventional optical 
disk tor high quality picture recording is not compatible 
with a standard video reproduction apparatus. Next, 
objectives of the present invention will be described A 
first objective of the present invention is for providing e 
optical -disk for high quality picture recording compatible 
with the standard video reproduction apparatus and a 
reproduction system for such an optical disk. 
J0004] The compatibility herein can be defined as 
the relationship between the conventional monaural 
records and stereo records. That is. a novel 3D optical 
disk or high resolution optical disk according to the 
present invention is output with a standard resolution by 
the existing reproduction apparatus for DVD or the like, 
and output with a high resolution by a novel reproduc- 
tion apparatus according tc the present invention. 
[0005] Then, a second problem of the conventional 
system is regarding a reproduction control system. By 
the conventional reproduction control system, one 
stream is reproduced using one decoder. Accordingly, 
in order to connect two streams of a high resolution sig- 
nal seamlessly- i.e., without stopping the movement of 
the video, a complicated system is required. A second 
objective of the present invention is for providing a 
reproduction control for connecting a plurality of 
streams seamlessly by a simple procedure. 

DISCLOSURE OF INVENTION 

[0006] An optical disk reproduction apparatus 
according to the present invention is for reproducing a 
signal -recorded on an optical disk. The optical disk has 
recorded thereon, at least a first video stream repre- 



senting a low iiequency component of the video signal 
and a second video stream representing at least a high 
frequency component of the video signal, the first video 
stream includes a plurality of lirst interleave units and 

i the second video stream includes a plurality of second 
interleave units, each of the plurality of first interleave 
units rncludes mi GOPs (where ml is an integer of l or 
greater), each ot the plurality of second interleave units 
includes m2 GOPs (where m2 is an integer of i or 

7 f greater). The optical disk reproduction apparatus 
includes a reproduction section tor reproducing the first 
video str earn and the second video stream recorded on 
the optical disk; a division section tor dividing the repro- 
duced first video stream into the plurality of first inter- 

75 leave units and for dividing the reproduced second 
video stream into the plurality of second interleave 
units; a decoding section for decoding the plurality of 
first interleave units to generate a first reproduction sig- 
nal representing the low frequency component of the 

po video signal and for decoding the plurality of -second 
interleave units to generate a second reproduction sig- 
nal representing at least the high frequency component 
ot the video signal; a synthesis section for synthesizing 
the first reproduction signal and the second reproduc- 
es tion signal to generate the video signal, and an output 
section for selectively outputting at least one of the first 
reproduction signal, the second reproduction signal, 
and the video signal. The above -described objective -is 
achieved by this. 

50 [0007] The plurality of first interleave units may be 
each corresponded to first time information relating to 
reproduction time, and the plurality of second interleave 
units may be each corresponded to second time infor- 
mation relating to reproduction time. 

35 [0008] The optrcal disk reproduction apparatus may 
further include a reference time signal generation -sec- 
tion tor generating a reference time signal; a first repro- 
duction control section for controlling the reproduction 
time ot the first reproduction signal in accordance with 

40 the difference between the reference time signal and 
the first time information; a second reproduction control 
section for controlling the reproduction time of the sec- 
ond reproduction signal in accordance with the differ- 
ence between the reference time signal and the second 

<5 time information; and an adjusting section for adjusting 
the reference time signal so that the reference time sig- 
nal supplied to the first reproduction control section and 
the reference signal supplied to the second reproduc- 
tion control section represent substantially the same 

sc time. 

[0009] The adjusting section may adjust the refer- 
ence time signal based on audio reproduction time 
information representing the time to reproduce an audio 
signal which is tc be output in synchronization with the 
£f video signal. 

[0010] The adjusting section may adjust the refer- 
ence time signal based on at least one of first video 
reproduction time information representing the time to 
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lepioduce the first reproduction signal and second 
video reproduction time information representing the 
time to reproduce the second reproduction signal. 
|0011] The lirst reproduction control section may 
control the i production time o1 the lirst reproduction 5 
signal by skipping a trame o1 the lirst reproduction sig- 
nal or by reproducing a trame ol the first reproduction 
signal in repetition. The second reproduction control 
section may control the reproduction time of the second 
reproduction signal by skipping a frame of the second w 
reproduction signal or by reproducing a frame of the 
second reproduction signal in repetition. 
J0012] At least one ol the first time information and 
the second time information may include at least one of 
a PTS. a DTS and an SCR. 75 
[001 3] The first r epi oduction signal may cor respond 
to a first pixel number, and the second reproduction sig- 
nal may correspond to a second pixel number, which is 
larger than the first pixel number. The synthesis section 
may include a converter for converting the first repro- 20 
duction signal into a conversion signal corresponding to 
the second pixel number. The video signal may be 
obtained by synthesizing the conversion signal and the 
second reproduction signal. 

[0014] The optical disk further may have, recorded 25 
thereon, an identifier representing the first pixel number 
corresponding to the first reproduction signal, and the 
converter may convert the first reproduction signal into 
the conversion signal in accordance with the identifier. 
[0015] The optical disk further may have, recorded 50 
thereon, an identifier representing the first pixel number 
corresponding to the first reproduction signal. The opti- 
cal disk reproduction apparatus may further include a 
rotation control section for controlling the rotation of the 
optical disk. The rotation control section may control the 5£ 
rotation of the optical disk in accordance with the identi- 
fier. 

[0016] The optical disk further may have, recorded 
thereon, an identifier representing that the video signal 
is obtained by encoding a progressive video signal of 24 <o 
frames to 30 frames per second. The output section 
may include a converter for converting at least one o1 
the first reproduction signal, the second reproduction 
signal, and the video signal into a frame signal. The out- 
put section may output the progressive video signal of <t 
60 frames per second by outputting the frame signal in 
an overlapping manner. 

[0017] The optical disk reproduction apparatus may 
further include a buffer memory section tor storing the 
plurality of first interleave units and the plurality of sec- so 
ond interleave units. The buffer memory section may 
have a capacity which is equal to or greater than an 
amount of data of the GOP or GOPs included in the sec- 
ond interleave units. 

[0018] The buffer memory section may have a ee 
capacity which is 1 MB or greater. 
[0019] An optical disk according to the present 
invention include, recorded thereon, at least a first video 



stream representing a low frequency component of the 
video signal and a second video stream representing at 
least a high frequency component of the video signal, 
wherein: the first video stream includes a plurality o! first 
interleave units, the second video stream includes a 
plurality of second interleave units, each of the plurality 
of first interleave units includes ml GOPs (where ml is 
an integer of 1 or greater), and each of the plurality of 
second interleave units includes m2 GOPs (where m2 is 
an integer of l or greater). The above -descr toed objec- 
tive is achieved by this. 

(0020] The plurality of first interleave units and the 
plurality of second inter leave units may be structured so 
that reproduction time of one of the plurality of first inter- 
leave units is substantially equal to reproduction time of 
one of the plurality of second interleave units, the one of 
the plurality of second interleave units corresponding to 
the one of the plurality of first interleave units. 
[0021] An optical disk recording apparatus accord- 
ing to the present invention includes a dividing section 
for dividing a video signal into a first video signal repre- 
senting a low frequency component of the video signal 
and a second video signal representing at least a high 
frequency component of the video signal: an encoding 
section for generating a first video stream by encoding 
the first video signal and for generating a second video 
stream by encoding the second video signals wherein: 
the first video stream includes a plurality of first inter- 
leave units, the second video stream includes a plurality 
of second interleave units, each of the plurality of first 
interleave units includes mi GOPs (where ml is an 
integer of l or greater), and each of the plurality of sec- 
ond interleave units includes m2 GOPs (where m2 is an 
integer of 1 or greater); a selection output section for 
selectively outputting the plurality of first interleave units 
included in the first video stream and the plurality of sec- 
ond interleave units included in the second video 
stream: and e recording section tor recording the signal 
output from the selection output section on an optical 
disk The above-described objective is achieved by this. 
[0022] The division section may include a decoder 
for decoding the first video stream and a differential cal- 
culator foi calculating a differential between the video 
signal and the signal output from the decoder, and may 
output the signal output from the differential calculator 
as the second video signal. 

[0023] The division section may further include a 
first converter for converting the video signal into a first 
conversion signal corresponding to a second pixel 
number which is smaller than a first pixel number corre- 
sponding to the video signal, and a second converter for 
converting the signal output from the decoder into a 
second conversion signal corresponding to the first pixel 
number which is larger than the second pixel number 
corresponding to the signal output from the decoder. 
The division section may output the first conversion sig- 
nal as the first video signal. The differential calculator 
may calculate the differential between the video signal 
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and the second conversion signal. 
[0024] The recording section may further lecord on 
the optical disk an identifier representing that the sec- 
ond vtfeo signal is output Horn the ditteiential calcula- 
tor 

(0025] The recording sea ion may further record on 
the optical cfcsk an oentrfier representing the tirst pixel 
number corresponding to the video signal. 
(0026] The recording section may turthei record on 
the optical disk an dentrhei representing the second 
pc*e! number corresponding to the tirst video signal. 
(0027] An opt>cai disk recording apparatus accord- 
ing to the present invention includes an input section tor 
receiving an encoded Ust video stream corresponding 
to a lust pi*e* rxmtwi and an encoded second video 
stream conebjjoncfrng to a second pixel numbei which 
is different from the **si f>*& 'XjnDei. wherein the first 
video stream includes * r>k/aMy o* f^st mierleave units, 
the second video st»oa*~ r>cxx»oi a plurality o1 second 
interleave units, cac* or r^r pk^aM, of trot mierleave 
umts includes ml GOH *«*^« «► an meoer ot 1 or 
greater), and each or r>» r^***» cr t+cond nierleave 
units includes m2 GC**\ % an *n*o«* ol 1 or 

greater): a seiectrm o/v** w^anr* w«*ovrty out- 
putting the plurality or t«w urwtt rxiuded in 
the first video stream and c*k*aifi o» vecond inter- 
leave unrts included n v*ccm roeo s*eam and a 
recording section k> reco»o>*x ugjnai output from 
the selection output or* mr optica} disk. The 
above <j esc f rbed objects % acrw«d b> r*s 
[0028] An optical on> »«c»oOuCt>on apparatus 
according to the present rw*cr a to reproducing a 
signal recorded on an optica' a*> T*>e optica! disk has, 
recorded thereon, at leas? « »rv *to>o stream including 
a pluralrty ot first GOPs ana a second video stream 
including a plurality or second ZwZ*** each of the plural- 
ity of first GOPs includes a ptsaaty ot pictures, and 
each of the plurality ot second ^+*s incJuoes a plurality 
ot pictures. The optica* cfc> teproduaion apparatus 
inciuoes a reproducton sec«r> io» reproducing the first 
vdeo-stream and the second **>to stream recorded on 
the optical disk, a decoo ng v^ctwn to* decoding the first 
video stieam and thtr s«*juno *o*o stream, and an out- 
put section foi selective * oavut**} the decoded first 
video stream and the o*cooad v*cond video stream in 
accordance with reproduction canto* mo'maiion. The 
reproduction control information viocaies that after e 
first picture included m a t«ar GDP among the plu- 
rality of first GOPs included «*~ tr>e trst video stream is 
reproduced, a second p«ctu» e nduded m a leading sec- 
ond GOP among the piuiaWy o» second GOPs included 
in the second video stream is t«>nioduced. the second 
picture being different from a u**nng picture of the lead- 
ing second GOF. The aoove-deso Oed objective is 
achieved by this. 

[0029] The decoding sector may start decoding 
the second video stream so that the decoding of the 
second picture has been completed when the reproduc- 



tion of the first picture is completed. 
[0030] The reproduction control information may 
include information tsl representing a position of the 
first picture, information ts2 lepiesentmg a position o1 

i the second picture, and information tsG representing a 
position of the leading picture of the leading second 
GOP. The decoding section may find a decoding start 
position ta in accordance with expression 
ta=tsi -(ts2-tsG) , and starts decoding the second video 

to stream based on the decoding start position ta. 

[0031] The reproduction control information may 
include timing information lepiesenting the timing to 
start decoding the leading second GOP so that repro- 
duction completion time ot the first picture matches the 

75 leproduction start time of the second picture. The 
decoding section may start decoding the second video 
stream based on the timing information. 
[0032] The decoding section may omit decoding of 
a picture which is not necessary for decoding pictures 

20 from the leading picture of the leading second GOP to 
the second picture. 

[0033] The picture which is not necessary may be a 
B picture. 

[0034] The optical disk reproduction apparatus may 
25 further include a buffer memory section for storing the 

first video stream and the second video stream, and the 

buffer memory section has a capacity which is .equal to 

or greater than an amount of data of 1 GOP. 

[0035] The optical disk has the reproduction control 
so information recorded thereon The reproduction section 

may reproduce the reproduction control information 

recorded on the optical disk. 

[0036] The optical disk may further have, recorded 
thereon, an identifier representing whether or not the 

35 reproduction control information is recorded on the opti- 
cal disk, and when the identifier represents that the 
reproduction control information is recorded on the opti- 
cal disk, the reproduction section may reproduce the 
reproduction control information recorded on the optical 

40 disk. 

[0037] In a fast reproduction mode, when the sec- 
ond picture is not an I picture, the output section may 
prohibit an I picture included in the leading second GOP 
from being output. 
<£ [0038] The output section may prohibit a part of the 
I picture included in the leading second GOP from being 
output based on I picture reproduction prohibition infor- 
mation. 

[0039] A reproduction control information genera - 
50 tion apparatus according to the present invention 
includes an input section for receiving a first video 
stream including a plurality ot first GOPs and a second 
video stream including a plurality of second GOPs; and 
a generation section for generating reproduction control 
£5 information which represents that after a first picture 
included in a final first GOP among the plurality of first 
GOFs included in the first video stream is reproduced, e 
second picture included in a leading second GOP 
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among the plurality ol second GOPs included in the 
second video stream is tepioduced. the second picture 
being differ errt from a leading piciuie ol the leading sec- 
ond GOP The above-described objective is achieved by 
this. 5 
(0040] The reproduction control infoimation may 
include information representing the number o1 pictuies 
from the leading picture of the leading second GOP to 
the second picture. 

[0041] The reproduction control infoimation may io 
include information representing the time to reproduce 
the leading picture of the leading second GOP and the 
time to reproduce the second picture of the leading sec- 
ond GOP 

(004?) The reproduction control information may is 
include timing mfoimation lepresenting the timing to 
start decoding the leading second GOP so that repro- 
duction completion time of the first picture matches the 
reproduction start time of the second picture. 
(0043) The timing information may represent the 20 
timing yo ittM decocfrng the leading second GOP when 
a p»ctL*» «**c^» r* not necessary for decoding pictures 
tro^ feadng pxr&*e of the leading second GOP to 

wrnmprtr* n. not decoded. 
(0O44] ntt^r* wTuch is not necessary may be a 25 

B per** 

[004*1 ±r appear o*> recording apparatus accord- 
mQtf» r*+nbon includes a generation sec- 

tion %> «v>q r oovoion control information; and 
a ••cxroo^ Mrtor o -ecoefcngthe reproduction con- 50 
tto» r*>***bor or «n opbcaJ Crsk having, recorded ther- 
eof « «.v rt*c tftem rKluding a plurality of first 
Gl**"» «ic * w*cond ^rvo stieam including a plurality of 
s*conr: :y**\ ««ooOuction control information 

t^jfwrmiM **** m »nj x^u>e included in a final lirst 55 
GD r among t-* pm»m> ol ttrst GOPs included in the 
ivy? m*c ur^am * eooouced. a second picture 
irojcMc r « iMOng wcono GOP among the plurality 
o* s^jonc ocxx)«d r» me second video stream is 

fepoaucac r»* w^canc petite being different from a <o 
leao^ng xv# o* r^- leacmg second GOP. The above- 
OebO t»c otjfvctrv* Ajr>tvvcJ by this. 
[0O44] A/i uutK-a* U«s> »eco»ding apparatus accord- 
ing to tne pi ever rvgrt on rcludes an editing section 
to edtmg a t« st vKjfo sd earn including a plurality of first «£ 
GOPs and a second *oeo stieam including a plurality of 
second GOPs so that at least one picture unnecessary 
fa reproduction it deleted m accordance with the repro- 
duction control infoimation. and a recording section for 
recording tne edited first video stream and the edited so 
second video stream on an optical disk. The reproduc- 
tion control informaton iepresents that after a first pic- 
ture included in a final first GOP among the plurality of 
first GOPs included in the first video stream is repro- 
duced, a second picture included in a leading second sb 
GOP among the plurality ol second GOPs included in 
the second video stream is reproduced, the second pic- 
ture being different from a leading picture of the leading 



second GOP The above-described objective is 
achieved by this. 

(0047] The at least one picture unnecessary tor 
reproduction may include a picture, of the first video 
slieam. after the first pictuie. and a picture, of the sec- 
ond video stream, before the second picture. 
(0048] The at least one picture unnecessary for 
reproduction may further include a picture which is not 
necessary for decoding pictures from the leading pic- 
ture ol the leading second GOP in the second video 
stream until the second picture. 

[0049] The at least one picture unnecessary tor 
reproduction may be a B picture. 

(0050] The recording section may record the edited 
f ii st video stream and the edhed second video stieam in 
continuous 1 egions of the optical disk. 
|0051 ] The recording section may record the repro- 
duction control information on the optical disk. 
(0052] The recording section may record the repro- 
duction control information on a medium other than the 
optical disk. 

BRIEF DESCRIPTION OF DRAWINGS 
[0053] 

Figure 1 is a block diagram of a 720P/480P hierar- 
chical recording apparatus in one example accord- 
ing to the present invention. 

Figure 2 is a block diagram of a 480i/480P/720P 
(60) reproduction apparatus in one example 
according to the present invention. 
Figure 3 is a block diagram of a 480P/720P (24/60) 
reproduction apparatus in one example according 
to the present invention. 

Figure 4 is a block diagram of a reproduction appa- 
ratus (720P output) of horizontal direction synthesis 
system in one example according to the present 
invention. 

Figure 5 is a block diagram of a three, hierarchical 
layer optical disk recording apparatus in one exam- 
ple according to the present invention. 
Figure 6 is a block diagram of a recording and 
reproduction apparatus of a frame-based reproduc- 
tion control system in one example according to the 
present invention. 

Figure 7 shows a process for recording steams by a 
recording apparatus of a reproduction control infor- 
mation recording system in one example according 
to the present invention. 

Figure 8 is a diagram comparing reproduction of an 
optica! disk in one example according to the 
present invention by an existing reproduction appa- 
ratus and by a reproduction apparatus according to 
the present invention. 

Figure 9 is a graph illustrating the relationship 
between the recording time period and the capacity 
of an optical disk in one example according to the 
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present invention 

Figure 10 is a block diagram ol a hierarchical iepio- 
duction apparatus o1 a 480P reproduction mode in 
one example according to the present invention. 
Figure 11 is a table showing a data stiucture o1 f 
lepioduction control information in one example 
according to the present invention. 
Figure 12 is a diagram illustrating a process lor 
recording a plurality o1 streams by a recording 
apparatus in one example according to the present io 
invention and a process toi reproducing the plurality 
ot streams by a reproduction apparatus in one 
example according to the present invention. 
Figure 13 is a flowchart illustrating reproduction 
control o1 two streams performed based on the 7* 
reproduction control information by a reproduction 
apparatus in one example according to the present 
invention. 

Figure 14 is a table showing a data structur e of the 
reproduction control information when the time ?c 
stamps of the streams are continuous in one exam- 
ple according to the present invention. 
Figure 15 is a diagram illustrating a process for 
recording and reproduction performed by a record- 
ing and reproduction apparatus in one example 25 
according to the present invention. 
Figure 16 is a flowchart illustrating a process for 
editing and generation of reproduction control infor- 
mation performed by a recording apparatus in one 
example according to the present invention. sc 
Figure 17 is a table showing a data structure of a 
picture identifier, representing information including 
resolution, of management information data in one 
example according to the present invention. 
Figure 18 is a block diagram showing an MPEG 35 
decoder of a reproduction apparatus of a different 
system in one example according to the present 
invention; 

Figure 19 is a diagram illustrating the principle o1 
multiple angle picture data division multiplex *o 
recording system in one example according to the 
present invention. 

Figure 20 is a diagram illustrating a method for 
recording horizontal and vertical interpolation infor- 
mation in interleave blocks after being divided in <t 
one example according to the present invention. 
Figure 21 is a view illustrating the principle of an 
M ADM system for dividing a signal into two in a hor - 
izontal, direction in one example according to the 
present invention. so 
Figure 22 is a diagram illustrating picture synthesis 
control performed by a reproduction apparatus in 
one example according to the present invention. 
Figure 23 is a diagram showing signal arrangement 
for outputting a progressive signal, an NTSC signal 55 
and a HDTV signal in one example according to the 
present invention. 

Figure 24 is a timing diagram of reproduction of pro- 



gressive. 3D and wide signals with respect to the 
data amount in buffer in one example according to 
the present invention 

Figure 25 is a block diagram of a reproduction 
apparatus in one example according to the present 
invention in an interlace video signal output mode. 
Figure 26 is a flowchart illustrating a method for 
performing AV synchronization of a first -decoder 
and a second decoder in one example according to 
the pr esent invention. 

Figure 27 is a flowchart illustrating a method for 
controlling two butter sections in one -example 
according to the present invention. 
Figure 28 is a timing diagram showing a data 
st i earn which is leproduced and output after proc- 
essed with buffering and decoding by the -decoder 
in one example according to the present invention. 
Figure 29 is a flowchart showing a detailed process 
for reproducing a program chain group by a system 
control section Mi -9 in one example according to 
the present invention. 

Figure 30 is a block diagram showing a structure of 
a part of an AV synchronization oontrol 12- TO, the 
part performing AV synchronization, in one exam- 
ple according to the present invention. 
Figure 31 is a block diagram of a data decoding 
section in one example according to the present 
invention. 

Figure 32 shows a signal format of a picture identi- 
fier in one example according to the present inven- 
tion. 

Figure 33 is a flowchart illustrating a process for 
STC switching for seamless connection in one 
example according to the present invention. 
Figure 34 is a view illustrating processing of a hori- 
zontal filter circuit in one example according to the 
present invention. 

Figure 35 is a block diagram showing a structure of 
an optical disk reproduction apparatus in one 
example according to the present invention. 
Figure 36 is a structural view of a video decoder in 
one example according to the present invention. 
Figure 37 shows a data structure of an optical disk 
in one example according to the present invention. 
Figure 38 is a timing diagram of video reproduction 
in one example according to the present invention. 
Figure 39 is a block diagram showing a structure of 
an optical disk reproduction apparatus in one 
example according to the present invention. 
Figure 40 is a structural view of an audiodecoder in 
one example according to the present invention. 
Figure 41 shows a data structure of an optical disk 
in one example according to the present invention. 
Figure 42 is a timing diagram of audio and video 
reproduction in one example according to the 
present invention. 

Figure 43 shows an optical disk reproduction appa- 
ratus in one example according to the present 
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invention. 

Figure 44 is a structural view ol a video decodei in 
one example according to the piesent invention. 
Figure 45 is a timing diagram o1 video i production 
in one example according to the preseni invention 
Figure 46 is a block diagram showing a stiuctuie ol 
an optical disk reproduction apparatus in one 
example according to the piesent invention. 
Figuie 47 is a structural view ol a video decodei in 
one example according to the piesent invention. 
Figure 48 is a structural view ot a video decodei in 
one example according to the present invention. 
Figure 49 is a structural view ot a video decoder in 
one example according to the present invention. 
Figure 50 is a block diagram showing a stiuctuie ot 
an optical disk reproduction appaiatus in one 
example according to the present invention. 
Figur e 51 is a structural view of an audio decoder in 
one example according to the present invention. 
Figure 52 shows a data structure of an optical disk 
in one example according to the present invention. 
Figure 53 is a timing diagram of audio and video 
reproduction in one example according to the 
present invention. 

Figure 54 is a timing diagram of operation frequen- 
cies of audio reproduction in one example accord- 
ing to the present invention. 

Figure 55 is a timing diagram of operation frequen- 
cies of audio reproduction in one example accord- 
ing to the present invention. 

Figure 56 is a diagram illustrating a flow of a stream 
in a reproduction apparatus in one example accord- 
ing to the present invention. 

Figure 57 is a flowchart illustrating a process for 
MPEG encoding, editing/reproduction control infor- 
mation generation, and reproduction control per- 
formed by a recording and reproduction apparatus 
in one example according to the present invention. 
Figure 58 is a block diagram of a recording and 
reproduction apparatus ot a frame-based reproduc- 
tion control system in one example according to the 
present invention. 

Figuie 59 is a diagram illustrating e process for 

deleting en unnecessary frame in one example 

according to the present invention. 

Figure 60 is a block diagram of a reproduction 

apparatus of a mutual authentication system and a 

TV monitor in one example according to the present 

invention. 

BEST MODE FOR CARRYING OUT THE INVENTION 

[0054] Hereinafter, the present invention will be 
described by way of examples with reference to draw- 
ings. 



(Example 1) 

(720P/480P hierarchical recoiding and reproduction 
system) 

(0055] With reference to Figuie 1 . a specific hierar- 
chical recording apparatus toi handling two hieiarchical 
layers ot 720P and 480P will be described. Later, a 
method tor recording a HDTV signal in a hieiarchical 
io manner in the state where the HDTV signal is divided 
into a plurality ol signals will be described, with letei- 
ence to Figure 20. 

[0056] In the case ot a movie signal, specifically, an 
oiiginal 720P video signal ot 60 frames pei second, the 

15 signal is input and then has extra frames deleted by a 3- 
2 pull-down section 746. As a resutt. a 720P (24 P) sig- 
nal 703 of 24 frames per second is obtained. In the case 
of a normal 60P video signal, the 3-2 pull-down section 
is bypassed. Herein. 60 P refers to 60 frames per sec- 

pg ond. The 720P video signal 703 having 1280 x 720 pix- 
els is processed by a 720P/480P down-converter 704 
as follows. First, the number of vertical lines is reduced 
to 720 x 2/3 = 480 by a vertical filter 705. Then, the 
number of pixels is reduced to 1280 x 9/16 s= 720 pixels 

25 by a horizontal filter 706. Thus, the 720P video signal 
703 is converted into a 480P video signal 707 having 
720 x 480 pixels. Such a low resolution 480P video sig- 
nal is encoded by an MPEG encoder 708 for 480P into 
a compression MPEG signal. Then, the compression 

so MPEG signal is decoded back into a 480P video signal 
710 by an MPEG decoder 709. This signal is enlarged 
to 3/2 times and 16/9 times respectively by a vertical fil- 
ter 712 and a horizontal filter 713 in a 480P/720P up- 
converter 711, and thus is converted into a 720P high 

3£ resolution video signal 71 4. The original 720P video sig- 
nal 703 and the 720P video signal 714 obtained by 
MPEG encoding and decoding are differential -calcu- 
lated by a calculation circuit 715 in a differential signal 
processing 720, and thus differential information 716 is 

40 obtained. 

[0057] The differential information 716 is encoded 
by a second MPEG encoder 717 for 720P into a GOP- 
based video signal including an intraframe (i picture) 
and a differential frame (P or B). This signal is divided by 

45 multiplex means 71 9 into GOP-based second interleave 
blocks 718a and 71 8b including 1GOP to nGOP. An 
MPEG stream of a basic signal encoded by the first 
MPEG encodei 708 for 480F in a basic signal process- 
ing section 721 is made into a 480P GOP-based MPEG 

so stream and then divided by the multiplex means 719 
into first interleave blocks 722a and 722b. The first inter- 
leave blocks 722a and 722b are interleaved into the 
second interleave blocks 718a and 718b; i.e., the first 
interleave blocks 722a and 722b and the second inter- 

55 leave blocks 71 8a and 71 8b are alternately arranged. 
The resultant signal is recorded on a disk 724 such as a 
DVD or the like by recording means 723. Also recorded 
at this point are a hierarchical recording identifier 725 
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indicating the start point and the teimination point and 
specified interleave block iepioduction piohibrtion infoi- 
mation 726 toi prohibiting the second interleave blocks 
718a and 718b including the ditleientia! irrtoimation 
trom being repioduced by the conventional iepioduction 
apparatus. The identrtiei and intormation aie lecorded 
m oveiall management information 224 and each oi 
VOBs as shown in Figure 23. 

{0058] When the disk 724 is repioduced by the 
existing reproduction apparatus based on the DVD 
standards as shown in Figure 8. the intei leave blocks 
722a and 722b are regarded as a tirst angle and repro- 
duced The reproduction signal is decoded by MPEG 
data 727. and thus NTSC or 480P (24 trames) video 
signal is repioduced As shown in Figuie 23, the speci- 
fied mie« leave block reproduction piohibition intorma- 
tion 726 to* prohibiting specified interleave blocks 
including me Differential information from being repro- 
duced *o* e*amp<e. an angle switching prohibition flag 
is recococa Accodmgiy even if the user inadvertently 
operatei. tr>* » •production apparatus, reproduction of a 
second an^« i * the tecond interleave unit is prohib- 
ited tn otf^#» wot*. r»* ential information for 720P 
is ai/nrwtrjiiy r§ m*+rr*rt fiom being reproduced by 
the eir&*xj rn/D ••sr*nrkjcton apparatus. When the dif- 
ferentia? rtjfrwBor to» 7?oP is reproduced in error, 
malfunaor* occ*#\ ux« e»s signal cannot be normally 
reproo\jL«c t>* a '«v MPfG decoder foi 480i of the 
existing; acxrtor ax'M'atus This type of trouble is 
avo<3*c rr>e p»rw^i rvfnton In this case, informa- 
tion or uxv^aor tc tr* w*cond interleave blocks may 
be irrer-r:ir nojXJ *-om the management infor- 
mat»or 22 a m^mcr « rr+t*-+c! tD as the navigation infor- 
matter 1 r- n*- r%1 iunu^ 

[0059] ">*e aexv* opvo ceo effect is also useful 
when * ?ro*> ugrw» e^** n> recorded in the second 
interleave oooi mm c^i* T>e 720 P signal is directly 
inpui tc M^t O «ncoa*» 717 as shown by the arrow 
indicatec " w> ) gut t 

[0060] r» tnts m^rv •wtven the disk 724 is repro- 
ducer D r tne e»fifcng O^'V »ep»oduCTion apparatus, a 
vioeo vgriAi rs> r«u» lxj^jl^o «£ the qualrty equivalent to 
NTSC wt»c*i •*> ulrt*'^ L>> itfp-oducinc an existing 
DVD 3<s* and tu- tr** rr»o» * e«oneous reproduction of 
information wfn»cr cannoi no-mally reproduced by 
the e*isi«ng DVD iep»oo\jc!ori apparatus, such as a dif- 
ferential signal o* a 7?0P s^gna 1 . is prevented. Thus, 
bidirectional compat*»i«r> n realised 
(0061] A 4B0P signal rtseit may be recorded in the 
second interleave blocks instead of the 720P signal. In 
this case, the lirst interleave blocks are reproduced and 
thus a 480i (NTSC) signal is output by the conventional 
reproduction apparatus By a reproduction apparatus 
according to the present invention, a 480i signal from 
the first interleave blocks or a 460P signal from the sec- 
ond interleave blocks is reproduced, oi both of them can 
be reproduced 

JO062] When a reproduction apparatus according to 



the piesent invention is used, a basic signal is repro- 
duced from the first interleave blocks 722a and 722b. 
which is reteued to as the first angle in the DVD stand- 
ards. A differential signal and a 720P signal are repro- 
5 duced fiom the second interleave, blocks 718a and 
718b, which is referred to as the second angle in the 
DVD standards From the first angle, a 480P video sig- 
nal 729 is output by an MPEG decoder 728 for 480P; 
and from the second angle, a 720P video signal 731 or 
to a 720P signal as a differential signal is reproduced by 
an MPEG decodei 730 foi 720P. These two video Sig- 
nals having a differenl numbei of pixels are synthesized 
by a synthesis section 732 oi output as they are, and 
thus decoded into an original 720P video signal 733 to 
7 £ be output. 

[0063] In this mannei, when the hierarchical record- 
ing disk 724 is i eproduced by the reproduction appara- 
tus according to the present invention, a 720 P video 
signal is output. Thus, a HDTV signal such as a 720P 
pc signal can be recorded while compatibility with the con- 
ventional reproduction apparatus is maintained. 
[0064] When a 480P signal itself is recorded in the 
second interleave blocks, a 480 P signal having a den- 
sity twice as high as that of an NTSC signal is repro- 
of duced. 

[0065] With reference to Figure 3. a more specific 
operation of the reproduction described with reference 
to Figuie 8 will be described- The blocks which have 
already been described will not be described again. 

30 [0066] The disk 724 has a basic signal and a differ- 
ential signal recorded thereon alternately after being 
divided on an nGOP-by-nGOP basis by the multiplex 
means 719 shown in Figure 1 and interleaved. This sig- 
nal is divided intc the first interleave block 722a and the 

35 second interleave block 71 8a, i.e., the basic signal and 
the differential signal, by a division section 734 of the 
reproduction apparatus shown in Figure 3. Then, the 
basic signal and the differential signal are stored in a 
first butter memory 735 and a second buffer memory 

<o 736 respectively. Then, respective time information is 
extracted from a time information extraction section 
793. A VTS synchronization section 780 sets first basic 
time intoimation and second basic time information in 
the first decodei 728 and the second decoder 730 so 

<£ that the two signals are in synchronization with each 
other. Thus, output signals from the two decoders are 
synchronized. In this case, when the hierarchical 
recording identifier 725 is detected, an identification 
information processing section 745 recognizes that a 

so first reproduction signal which is the decoded signal of 
the firs: stream is a basic signal having a smaller 
number of pixels and that a second reproduction signal 
which is the decoded signal of the second stream is a 
differential signal having a larger number of pixels and 

5f having the differential information trom the basic signal. 
Thus, the identification information processing section 
745 give the synthesis section 732 an instruction 
regarding an up-converter 738 and an instruction of 
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addition 

10067] The MPEG decodei 728 ioi 480P and the 
MPEG decodei 730 lespeclively decode the signals 
into a 480P (24) signal and a 720 (24 frames) signal. 
The decoded signals have 24 ttames/sec. oi 30 
frames/sec. The signals are processed by 2 3 conver- 
sion sections 737a and 737b so as to output the same 
Irame twice, and thus a 480P signal 729 oi 60 
frames/sec. and a 720P signal 731 having the differen- 
tial infor mation are obtained. The 480P signal 729 is up- 
converted into a 720P signal 739 by the 480P/720P up- 
converter 738 and added to the 720P signal 731 having 
the differential inlormation by an addition section 740. 
and thus the original 720P video signal 733 is obtained. 
The addition section 740 calculates, foi example, as 
shown in the figuie. Wheie the pixels oi the respective 
signals are a and b. (a+b)/2 is performed to obtain the 
original 720P video signal 733. The calculation per- 
formed by the synthesis section 732 may be different 
from (a+b)/2. 

[0068] In this case, the MPEG decoding signals 
may be kept to have 24 frames/sec. without being con- 
verted by the 2-3 conversion sections 737a and 737b to 
have 60 frames/sec. and after synthesis, converted to 
have 60 frames/sec. by a 2-3 conversion section 741 . In 
such a case, the amount of data of the video signal is 
advantageously reduced to half, and the processing 
ability of the digital processing circuit can be reduced to 
half. 

[0069] With reference to Figures 1 and 3, the 
method for recording and reproducing a 720 P signal of 
24 frames/sec. such as a movie signal in a hierarchical 
manner has been described. This method has signifi- 
cant advantages. The HDTV format includes the I080i 
system and the 720P system. As shown in Figure 9. a 
I080i signal (24 frames) used for movies can be 
recorded only tor 90 minutes since a two-layer DVD has 
a capacity of 6.5 GB as indicated by curve 742a. 
[0070] By contrast, a 720P signal (24 frames) can 
be recorded for 150 minutes as indicated by curve 
742b. A 480P signal (60 frames) can be recorded for 
150 minutes as indicated by curve 742c. Disks for mov- 
ies are considered to be meaningless unless each has 
a recording capacity of 120 minutes or more. The 720P 
(24)/480P hierarchical recording disk has an effect that 
an HDTV movie title can be accommodated in one DVD. 
[0071] In the example shown in Figure 3, a 480P 
signal having the basic information of a 720P signal is 
recorded in the first interleave blocks, and the differen- 
tial information between the 720P and 480P signals is 
recorded in the second interleave blocks. For a disk 
having a 720P signal as rt is in the second interleave 
blocks is reproduced, the output from the second 
decoder 730 can be output as it is as shown by arrow 
indicated with mmm in Figure 3. The determination to do 
this is made by the identification information processing 
section 743 based on an identifier. In this case also, an 
effect equivalent to complete compatibility is obtained. 



This system has a lower recording efficiency but has the 
effect of simplifying the processing circuit for recording 
and reproduction and the effect of the complete compat- 
ibility. 

5 [0072] Wrth reference to Figure 60, an example in 
which a decoder is mounted on a TV monitor 798 will be 
described. The basic operation is the same as that 
descr ibed with refer ence to Figure 3, and only the differ- 
ent pans will be described. In a reproduction apparatus 

io 743a. a signal before decoding is encrypted by an 
encryption encoder 795 using an encryption key 799a 
and sent from a communication interface section 796a 
through a network 798 to a communication interlace 
section 796b in the TV monitor 798. Before this opera- 

75 tion. mutual authentication sections 794a and 794b 
communicate to each other to authenticate each other. 
This operation may be refened to as a handshake. In 
the case where mutual authentication is confirmed and 
thus both sides determine that the other side is proper. 

20 the mutual authentication sections 794a and 794b pro- 
vide the encryption encoder 795 and an encryption 
decoder 797 with the encryption keys 799a and 799b 
and also permit the communication interface 796a and 
796b to communicate. Thus, encryption data is sent 

25 and received, and the keys of the encryption data are 
unlocked. Therefore, the first stream and the second 
stream are respectively sent to the first decoder 728 
and the second decoder 730. The determination to con- 
duct this processing is made by the identification infor - 

50 mation processing section 745 based on an identifier 
744 which is separately sent. When the first stream is of 
a 480P signal and the second stream is of a 720P differ- 
ential signal as described above, up-conversion and 
synthesis calculation are performed, and a 720P signal 

35 is output to a TV monitor 798a. When an identifier indi- 
cating that the second stream is of a 480P differential 
signal is received, the two streams are synthesized to 
output a 480P signal When an identifier indicating that 
the streams are of a 3D signal, a 3D signal is output and 

40 displayed on the TV monitor 798a. The 3D signal is 
obtained by time-based synthesis, where the first 
stream is set for the left eye and the second stream is 
set for the right eye. 

[0073] According to this system, even when two 
<s streams are encrypted, the streams are processed, for 
example, synthesized by the TV monitor using the iden- 
tifier 744. Thus, the original picture can be obtained 
without violating the copyright protection secured by the 
encryption. 

so [0074] Next, with reference to Figure 10, an opera- 
tion for reproducing a disk 724a having a 480P <60 
frames/sec.) recorded thereon by the reproduction 
apparatus according to the present invention will be 
described. Common parts as those in Figure 3 will not 

be be described. 
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{Sum and difference system - Figuie 19) 

{0075] With reference to Figuie 19. the concert ol 
the sum and difference system will be described. This 
system is referred to as multiple angle video data divi- 
sion multiplex system (MADM) since a video signal is 
divided into vertical or horizontal high Irequency and low 
frequency components and recorded in each ol multiple 
angles. As shown in Figure 19. a signal is divided into a 
basic signal (sum signal) and a sub signal (differential 
signal) by a sum calculation section 141 and a drffeien- 
tial calculation section 143. The resultant signals are 
MPEG-encoded and then alternately recorded as inter- 
leave blocks in units of 1 GOP. At this point, the amount 
of the information can be reduced by 20% by perfoi ming 
a 3-2 conversion of the basic signal and the sub signal 
in synchronization with each other. It is efficient to use. 
as the basic signal, "IBBPBBPBBPBBPBB*" which is 
shown as a main GOP structure 244 used lor the ordi- 
nary MPEG encoding. In this structure, an I fiame 246. 
B frames 248 and P frames 247 are alternately 
arranged. In the case of the differential signal, however, 
experiments have shown that it is efficient to have a 
structure including only I frames 246 and P frames 247 
due to the profile pattern, for example. "IPPPPPP- 
PIPPPPPPP- shown as a sub GOP structure 245. The 
efficiency is improved by changing the setting for the 
sub GOP structure. 

(0076] Figure 19 shows an example in which a 
480P video signal is divided into two in a vertical direc- 
tion. 'Figure 21 (described below) shows an example in 
which a 480 P video signal is divided into two in a hori- 
zontal direction. In an alternative manner, a 60-frame 
480P signal may be divided by frame division means 
into 30 odd frames and 30 even frames. In this case, the 
respective 30P signals are converted into two 60-field 
interlace signals, and each of the signals are MPEG- 
encoded and recorded in the MADM system. Such 
encoding is performed in a progressive manner, and 
therefore encoding efficiency is improved as in the case 
of the movie. Thus, the recordable time period of the 
same disk is extended 

(0077] When such a signal is reproduced by a non- 
MADM reproduction apparatus, a 30P (one-channel) 
525 interlace signal is reproduced in a first channel. 
Such a signal lacks necessary frames end is distorted. 
[0078] When such a signal is reproduced by an 
MADM reproduction apparatus, a 30P signal is repro- 
duced as a basic signal and another 30P signal is repro- 
duced as a sub signal. These two 30-frame signals are 
synthesized into a 60 -frame normal 480P signal by 
frame synthesis means including a frame buffer, and 
then output. 

[0079] When a line doubler is added to an output 
section for the 480P signal, a T050P video signal is 
obtained. 

{0080] When a 525 interlace signal is input to a sum 
sicnai section of the synthesis section of the MADM 



repiodudion apparatus and the value of 0 is input to a 
drfteiential signal section of the synthesis section, a 
480P video signal is obtained. Such a mannei ot input 
has the same effect as the line douWer. This method 

i allows even a 525 interlace signal to be output as a 
480P signal Accordingly, all types ot pictures can be 
viewed by simply connecting one cable to a progressive 
input terminal ot the MADM reproduction apparatus 
[0081] In Figure 19. 1/2(A+B) and V2(A-6) aie 

io used as expressions for calculation for a two- tap. filter. 
The division tiequency corresponds to about 300 scan* 
ning lines. 

[0082] As shown in Figure 19. the disk 724a having 
a 480P signal in the state of being divided into two sig- 

i5 nals by sum and differential calculations and recorded in 
two block groups, i.e., the first interleave blocks and the 
second interleave blocks is reproduced. The signal is 
divided into a 480 i signal as a basic signal and a 480i 
signal as a differential signal by the division section 734 

?c The signals are respectively decoded by the MPEG 
decoders 728 and the MPEG decoder 730 to obtain a 
480i signal 729a and a 480i differential signal 731a. The 
addition section 740 performs the calculation of (a-i-b)/2 
to synthesize the two 480i signals Thus, a 480P (60 

25 frames) synthesis signal 733a is output. 

[0083] The disk 724 includes 480i/480P/720P iden- 
tification information 744 (Figure 17) recorded in a toe 
section or the like thereof. The 480i/480P/720P identifi- 
cation information 744 includes thr ee signals in the case 

so of the 480i signal, the 480P signal and the 720P signal, 
and also shows which resolution of differential signals 
are recorded. The identification information processing 
section 743 processes this information to determine in 
which sector address ol the disk main data (main signal) 

5f and sub data (differential signal) of the hierarchical data 
are recorded. The identification information processing 
section 743 then sends information on the start point 
and the like to the synthesis section 732. The synthesis 
section 732 performs a synthesis calculation of the main 

<o data and the sub data from the start point of the 480P 
signal and outputs a 480P (60 frames/sec.) signal. 
[0084] As shown in Figure 17, row of Vts=6, it is 
recorded on the disk that at the start point of the 720P 
signal, 720P-main is the first interleave block, and 

<e 720P-sub is the second interleave block. The identifica- 
tion information processing section 743 identifies this 
information. The calculation section 740 performs a 
synthesis calculation of the 720P signal, for example, 
<a+b)/2 from the start time stamp of the 720 P signal, 

56 using the time stamps of the main signal and the differ- 
ential signal from the MPEG decoders 728 and 730, and 
then outputs a 720P signal. 

[0085] When a 480P identifier is recorded as the 
identification information 744 (Figure 17). the iderrtrjica- 
££ tion information processing section 745 sends a 480i 
signal decoding instruction to the MPEG decoder 730 
and causes the MPEG decoder 730 to decode the 480i 
signal. Then, a differential signal 73ia of the 480i signa! 
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is decoded and synthesized by the synthesis section 
732. Thus, a 480P (60 frames/sec.) is output. 
[0086] In this mannei. the MPEG tiecodei 730 per- 
forms the 480 i processing (480P-30 frames/sec.) oi the 
720P processing in accordance with the identification 
inlormation. Thus, the main signal and the differential 
signal o1 the 480i signal, and the main signal and the 
sub signal erf the 720P signal, can be decoded by two 
MPEG decoders in total. This has the effect oi simplify- 
ing the structure of the apparatus. 
{0087] The 480P reproduction mode shown in Fig- 
ure 10 does not use the 480P/720P up-converter 738 in 
the synthesis section 732. but allows the decoded 480P 
(60) signal to be up-converted to a 720P signal by the 
480P to 720P up-converter 738 and thus displayed on a 
HD video projector tor 720 P oi the like. Thus, the scan- 
ning lines are advantageously more unlikely to be 
viewed. In this case, one 480P to 720P up-converter 
738 can be used for 720P signal synthesis and 480P to 
720P up-conversion. Thus, the 480P signal can advan- 
tageously be up-converted into a 720 P signal without 
adding any element. 

(720P/480P/480i three hierarchical layer r ecording 
apparatus) 

[0088] With reference to Figure 5. a structure and 
an operation of the 720P (60 frames/sec. )-type three 
hierarchical layer recording apparatus will be described. 
The structure and the operation are substantially the 
same as those in Figure 1 . and only the different parts 
will be described. First, an input signal is a 720P signal 
of 60 frames/sec. Accordingly, after being 480P down- 
converted, the input signal is a 480 P signal of 60 
frames/sec. This signal is input to a basic signal 
processing section 721 a and processed by a division 
section 747. Where the pixel data of the n'th line is a and 
the pixel data of the (n+1 )th line is b. the division section 
747 uses the calculation result ot (a+b)/2 for the m'th 
line of a 480i video signal 748a and the calculation 
result of (a-b)/2 for the m'th line of a 480i video signal 
748b, and thus obtains a main signal and a sub signal 
of an NTSC signal. These signals are respectively 
encoded by the MPEG encoders 708a and 708b, and 
decoded by the MPEG decoders 709a and 709b into 
decoded signals 749a and 749b. The signals are syn- 
thesized by a synthesis section 748 into a 480P signal 
710. The 480P signal is up-converted into a 720P signal 
714, and differential information is obtained. The differ- 
ential information is MPEG-encoded to obtain data as 
third interleave block data 71 8a and 718b. This proce- 
dure is substantially the same as that in Figure 1 except 
that the frame rate is 60 frames/sec. instead oi 24 
frames/sec. 

[0089] The 480i MPEG stream is divided by multi- 
plex means 719a into interleave blocks on an nGOP 
basis. The nGOPs are interleaved in the order from a 
480i signal (first interleave blocks 722a of a basic sig- 



nal), then a 480i signal (second interleave blocks 750a 
of a differential signal), and then a 720P (third interleave 
blocks 718a of a differential signal), and recorded on the 
disk 724 such as a DVD. 

s [0090] In this case, the multiplexed signals are 
modulated by an 8VSB. OAM or OFDM modulation sec- 
tion 751 and transmitted from a transmission section 
752. Thus, hierarchical broadcasting can be performed. 
The signals may be multiplexed by time division based 

to on a time domain denned by the broadcasting instead of 
based on a GOP 

[0091] In this manner, a 480i/480P (60)/720P thiee 
hierarchical layer disk or hierarchical broadcasting is 
realized. 

15 [0092] With reference to Figure 2. an operation for 
reproducing the disk 724a will be described. Since iden- 
tical elements with those in Figure 3 are included, the 
identical elements will not be described. A signal repro- 
duced from the disk 724a or received by a receiving 

20 section 753 and demodulated by a demodulation sec- 
tion 754 is divided by the division section 734 into three 
streams based on the above-mentioned interleave 
blocks. The three streams are decoded by the three 
MPEG decoders 728a. 728b and 730 through buffers 

25 735a. 735b and 736. Then, three signals, i.e.. the 480i 
signal 749a. the 480i differential signal 749b. and the 
720 P differential signal 731 are obtained as a result of 
demodulation. By subjecting the 480i basic signal 749a 
and the 480i differential signal 749b to the calculations 

so of (a-i-b) and <a-b) by a synthesis section 732, a 480P 
(60 frames/sec.) video signal 729 can be obtained. This 
signal and the above-mentioned 720P differential signal 
731 are synthesized into a 720P output 733a. The pro- 
cedure is described above and will not be repeated. 

5£ [0093] In this manner, three types of outputs of 480i 
output 749a, 480P output 729 and 720P output 733a 
having different resolutions can be obtained from the 
disk 724a. The user can select the output by the grade 
of the monitor reproduction apparatus. That is, the 480i 

40 (NTSC) grade output is obtained by the existing repro- 
duction apparatus, the 480P (60 frames/sec.) is 
obtained by the reproduction apparatus for 480P 
according to the present invention, and the 720P (60 
frames/sec.) is obtained by the reproduction apparatus 

<5 for 720P according to the present invention. Thus, the 
complete compatibility is realized. 
[0094] In Figure 2, when the identification informa- 
tion processing section 745 detects a high resolution 
identifier, the rotation rate of the motor is raised through 

so a system control section 21 and a rotation control circuit 
35. A high resolution signal can be reproduced by rais- 
ing the rotation rate in accordance with the identifier. 
The rotation rate is raised to IX for reproduction of a 
standard picture. 2X for 480P and 720P (24 P). and 3X 

ee to 4X for 720P (60P). The effect of power saving is pro- 
vided. Then an NTSC grade signal is reproduced, the 
system control section 21 stops or operates, at a low 
rate.'clocks of the 720P MPEG decoder 730, the 480i 
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MPEG decode! 728b. and the synthesis section 732 
which aie not necessary. Thus, powei consumption can 
be significantly reduced Then an ATPS 84 ol the audio 
time stamp of audio data is received by an AV synchro- 
nization control section 158 and a video piesentation 
time stamp VPTS toi each ot the MPEG decodeis is 
created based on the time information and set in legis- 
ters 39a. 39b and 39c of the decodeis. frames can be 
synchronized tor lepioduction tiom the decodeis In 
ordei to synchronize the vertical blanking, e decoder 
synchronization section 794 simultaneously lesets the 
horizontal and vertical synchronization of the decoders 
The pictuies from the decoders can be synchionized on 
a dot-by-dot basis. Specific synchronization methods of 
audio and video signals will be described later. 
10095] Fiom the disK 724a, a lit si lesolution identi- 
fier indicating a low resolution of NTSC signals or the 
like of the picture o1 the first stream and a second reso- 
lution identifier indicating a high resolution of 720P sig- 
nals or the like of the second and third streams are 
reproduced. The system control section 21 determines 
by calculation which processing is to be performed by 
the up-converter 738 in the synthesis section 732 
among 480P to 720P, 480P to I080i. 480P to 1080P, 
and 720P to 1080P. and indicates the result to the syn- 
thesis section 732. In actuality, various first resolution 
identifiers ^xist such as. tor example. 704 x 480 and 720 
> 480. This has an effect that the up-convertei operates 
at the optimum ratio. Neediess to say. a simple system 
structure in which an identifier indicating the ratio of the 
up<onverter is recorded and reproduced can be 
adopted. 

10096] The reproduction apparatus 743a in Figure 2 
can output three resolutions of signals (i.e., 480i 
(NTSC) signal of the first stream. 480P (60P) signal 729 
of the first and second streams, and 720P (60P) signal 
733a of the first, second and third streams) simultane- 
ously or at different timing. Such a reproduction appara- 
tus can be used with monitors having various 
resolutions. 

{0097] Especially since a 480P signal 729 can be 
converted into a 720 F signal by the up-converter 738 of 
the s>nthesis section 732, the 720P signal obtained as 
a result of conversion o1 the 480P signal can be 
obtained without adding any circuit. 
[0098] In the case where a receiving section 753 
and a demodulation section 754 are added to the hier- 
archical reproduction apparatus, a receiving apparatus 
for receiving a hierarchical signal such as a TV signal, 
demodulating the signal and outputting three resolu- 
tions of video signals can be provided. 

(Wide 480 P) 

[0099] With reference to Figure 21 . a concept of the 
MADM system in which a signal is divided in a hori2on- 
tal direction is shown. The signal can be converted by a 
3-2 conversion section 174 into a 1440 * 480i interlace 



signal. The signal is divided into two in a horizontal 
diiection by a hoiizontal filtering section 206a. The prin- 
ciple of f iheiing is shown in parts (a) and <b) ol Figure 
34. As shown in part (b), 1440 dots are divided into odd 

t dots 263a and 263b. and even dots 264a and 264b. 
Where the odd dots are labeled as Xn and the even dots 
are labeled as Yn. a sum signal is obtained by thecal* 
culation of X-»Y and a differential signal is obtained by 
the calculation of X-Y. As a lesult. two 480P or 525i sig- 

7 f. nals. each of 720>480. are obtained as shown in part 
(b) of Figuie 34. 

[0100] Returning to Figure 21. the number ot hori- 
zontal dots of such a horizontal sum signal is reduced to 
720. Since the signal is passed through the horizontal 

i5 filter, howevet. aliasing distortion is as low as that of an 
NTSC signal. A conventional lepioduction apparatus 
reproduces only the sum signal and accordingly pro- 
vides a DVD picture of exactly the same quality. The dif- 
ferential signal represents only a profile formed of line- 

20 drawing. However, since the difference signal is 
restricted by a second video signal output restriction 
information provision section 179 { Figure 60) so as not 
to be reproduced by an ordinary reproduction appara- 
tus, no problem occurs The sum signal and the differ - 

?5 errtial signal are respectively encoded into MPEG 
streams by a first encoder 3a and a second encoder 3b. 
and subjected to interleaving in units of an interleave 
block of 1 GOP or more and MADM-multiplexed. 
[0101] In the case of the movie, 3-2 conversion is 

so performed by the 3-2 conversion 174 section and 
MADM-recorded as an MPEG signal together with the 
3-2 conversion information 174. 

[0102] In the case of the movie. 24 frames are 
reproduced in one second Accordingly, a 1440 x 480P 

55 progressive picture is reproduced based on two inter- 
lace signals by a 2X reproduction apparatus. The scope 
size of the movie is 2.35:1 The format of l440x480P is 
suitable for the scope size of 2.35:1 in terms of the 
aspect ratio. Thus, a wide screen 480P is effectively 

<o reproduced. 

[0103] A wide 480i hierarchical disk 724b is 
described with lefeience to Figure 21 above. With refer- 
ence to Figure 4, an operation for reproducing the<Jisk 
724b by a W-480i reproduction apparatus will be 

45 described. When the disk 724b has information 
recorded at 24 frames/sec, a W-480P basic signal 757a 
and a W480P differential signal 757b are obtained by 
decoding performed by field frame conversion sections 
756a and 756b. Each pixel is encoded by data obtained 

so by (X+Y)/2 or (X-Y)/2. Accordingly, when a synthesis 
section 758 performs the calculation of 
(X+Y)/2+(X-Y)/2 . X. i.e., data of odd pixels is obtained 
by the decoding. When the synthesis section 758 per- 
forms the calculation of (X* Y)/2-(X-Y)/2 . Y. i.e.. data of 

££ even pixels is obtained by the decoding. As a result, the 
number of pixels in the horizontal direction is doubled to 
1440 pixels. In this manner, a W480P video signal 759 
of 1440 x 480P pixels is obtained. The W480P video 
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signal 759 of 1440 > 480P pixels is converted by a 
W480P-720P conversion section 760 so as to have 
1280 puels m the horizontal dnection using a 6/9 hori- 
zontal littei 760a and so as lo have 720 pixels in the 
ve«ticai d«iection using a 3/2 vertical liHei 760b. As a 
result, a 7?0P digital output is obtained Thus, use ot a 
general 7?0P digital interlace is advantageously 
allowed 

(Detailed reproduction operaton F»guie25) 

I0104J Wrtn »eie«ence to Figure 25. an operation ol 
a teptoouction apparatus 65 according to the present 
m/emon t** CVso iDed m detail Figure 25 is a block 
dug* am ot rf it*>vjuuctun apoaiatus toi lepioducing a 
2X p«ou/ f^s***- u» -wxJtr pKTtuie o* 720P signal. A 

s»gnai r epi oouc ♦> cm an opt«oai disk 1 is divided by a 
division section 66 «**c iot^fi©ave blocks 66 and 
second inie«ioavt' tvocKi 67 eacf*> rxUudmg a frame sig- 
nal o« 1 GO p o rno « » lynr *ooo u»gnafc, 70a and 70b 
each ol 30 tiam«^-v»«: att^^rr a t*wj*t of MPEG 
e>tens»on pe*X> rr^rt t> *~ t«v>wcr v*et>on 69 are 
respectively (J vOhg r»% (Kw v^noos 71 a and 
71 b ntc onn i***ri v^w* r?» a -r? 7?t> «rrj even field 
signals 73a a nri 71b ? ^~w»ce signals 

74a and 74b aie autx* v.^****" iMown in Fig- 

ure 20 will be oeior*: l§*» ^-ng to figure 25. 
usmg the methoc O^u. t*: * i**o • 960 pro- 

gressive p«ctuie 182 m * o+*->m2 r * ho Lrcma! dir ection 
usmg a sub band fine* o cxr>re*von by a hori- 

zontal and vertical Ovrvor v»;t>on 194 in a picture divi- 
sion section 115 Tfvr.. * *>-^* p»o<yessive video signal 
183 is obta-ned lha t*gn* 0*0*0 mto 525 interlace 
signals 184 and lecctxo a. tr*^rrt 188a and the like. 
{0105] Remaning rtf» potatoes information 185 is 
divided similarly into tour 188c I88d, 188e and 

1881 and recorded as mif»Mvt b»ooa The maximum 
transiei rate 01 eacr rre- uoo rs E Mbps by the 
DVD stancards Accoc*ng» tne interpolation 

imormation is divided mtc tx> u> earns, the transfer rate 
is 3? Mops Wher tr*» tr»«wpota:ion information is 
divided into s» anpeu ir* t^rvref rate is 4& Mbps. 
Thus, a 720F c* ObO^ < T\ *oeo signal can be 
recorded By the co^^rit^jrw '^oduction apparatus, 
the stream 188a rs »n>ocvjk*c 10 outx/i an interlace 
video signal 184 Regywo^ m earn- 188c. I88d, 
I88e and 1881 outpus «ftt<3ior mfotmation is 
recorded on an optical ot> i£7 b> a pau^e processing 
restriction information geneiato" f»ect>on 179 There- 
fore, the interpolation mio» matior- 165. such as differen- 
tial information or the like w**c*"» »s not properly viewable 
is prevented from martvextprtty tx^ng output. By divid- 
ing the signal in hori?om»i arm vesical directions by the 
system shown in Figure 25 an optical disk compatible 
to both the HDTV and NTSC fc>mats is advantageously 
realized. 

[0106] In Figure 25. the mae«iace signal obtained by 
the conversion performed b> an interlace conversion 



section 175 is output to provide a scope screen 178. A 
480 P progiessive signal is similarly output on a scope 
scr een 178 When a monitoi lor 720P is used, a 480P 
signal is converted into a 720P progressive signal by a 

5 480P/720P conversion section 176. and as a result, is 
output on a 1280 > 720 or 1440 x 720 letter box type 
720P screen 177 (the picture has 1280 * 480 or 1440 x 
480 pixels). Since the scope picture (2.35:1) has 1 128 x 
480 pixels, a pictuie having a size closer to the aspect 

to ratio is obtained. Especially tn the case ol a movie, the 
signal is of 24 frames/sec. and so the progressive pic- 
ture is transferred at the rate of 4 Mbps. When a scope 
picture is recorded by the system according to the 
present invention of dividing the picture into two 

75 screens, the transfer rate is 8 Mbps. In such a case, 
about 2 hours of information can be recorded on a two- 
layer DVD. Accordingly, a 720P or 480P high quality pro- 
gressive picture signal for scope screen can be advan- 
tageously recorded on one DVD. On a conventional TV 

20 screen, the picture is displayed by an interlace output 
signal, needless to say. The present invention has an 
effect that a scope screen picture (2.35:1) of the movie 
can be output as a 480 P or 720P signal. 

25 (High resolution recording identification information) 

[01 07] Returning to Figure 1 . address information is 
output from an address circuit. A hierarchical recording 
identifier 725 including progressive/3D picture arrange- 
so ment information is output from a progressive/3D pic- 
ture arrangement information output section 725a. 
These pieces of information are recorded on the optical 
disk by a recording circuit 723. The progressive/3 D pic- 
ture arrangement information includes an identifier 
35 which indicates whether or not a progressive or 3D pic- 
ture is present on the optical disk, the hierarchical 
recording identifier 725 which indicates whether or not 
the signal is up-converted when being hierarchy- 
encoded, or a progr essive/3D picture arrangement table 
40 14 shown in Figure 17. As shown in Figure 17, a TEX- 
TDT file 83 includes, for each VTS, 3D pictures for the 
right and left eyes and angle numbers and cell numbers 
in which the progressive signal is located. Since a PGC 
file of each VTS includes a start address and a termina- 
ls tion address of each cell, the start address and the ter- 
mination address of each cell are included in the 
progr essive/3D picture arrangement information. Based 
on the arrangement information and identification infor- 
mation, the reproduction apparatus outputs a progres- 
50 sive picture or a 3D picture correctly as progressive 
outputs or R and L outputs. When ordinary pictures of 
different contents from each other are output as R and L 
outputs in error, the user will feel uncomfortable since 
the pictures for the right eye and the left eye are not 
55 related to each other. The progressive/3D picture 
arrangement information, the progressive/3D picture 
identifier, or the hierarchical recording identifier has an 
effect of avoiding the output of such unpleasant pic- 
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lures As shown in Figuie 3. when the hierarchical 
recording identifier 725 is reproduced, the contiot sec- 
tion sends an up-conversion instiuction 786 to up con 
vert the a 4B0P signal into a 720P signal by the up- 
converter 738. Then, the synthesrs o1 the 720P signal is 
performed. When the hierarchical recording identrtiei 
725 is not available, a 480P signal is output attei per- 
forming synthesis calculation without using the up-con- 
verter 738 as shown in Figure 10. In this manner, stable 
picture synthesis can be performed simply by switching 
the connection in accordance with whethei oi not the 
identifier is available. 

[0108] With reference to Figuie 23, s piocess toi 
performing reproduction using a pictuie identifier 222 
will be described. 

40109] From the optical disk, lepioduction piocess 
control information 225 is first read from the manage- 
ment information 224. Since the information 225 
includes restriction information on VOB, a 0th VOB 
226a is only connected to a first VOB 226b having a 
main picture by an existing reproduction apparatus. 
Since the Oth VOB 226a is not connected to a second 
VOB 226c having an interpolation signal such as differ- 
ential information or the like, an ugly picture such as dif- 
ferential information is prevented from being output by 
the existing reproduction apparatus as described 
above. Each VOB of the main signal has a picture iden- 
tifier. Since the progressive identifiers i and resolution 
identifier =00 (525) in the first VOB 226b and the second 
VOB 226c. a progressive signal having 525 scanning 
lines is reproduced from a progressive or HD reproduc- 
tion apparatus. 

[0110] In a picture identifier 222 of the next VOB 
226d, the progressive identifier =0 and the resolution 
identifier 219=10. This indicates that an interlace signal 
having 1050 scanning fines is output and that three 
VOBs 226e. 226f and 226g are interpolation informa- 
tion. Thus, an NTSC signal is output by a conventional 
reproduction apparatus, an interlace signal having 720 
horizontal pixels and 1050 vertical pixels is output by a 
progressive reproduction apparatus, and a full HDTV- 
format signal having 1050 scanning lines is output by a 
HD reproduction apparatus. As can be appreciated from 
this, various video signals can be recorded in an inter- 
leave manner and reproduced by the picture identifier 
222. The picture identifier 222 can be recorded in the 
management information 224. 

(2X clock and soft-decoding) 

[0111] In the block diagrams shown in Figures 3 
and 4. two MPEG decoders are used. In Figure 18. s 
first MPEG signal and a second MPEG signal are syn- 
thesized into one MPEG signal by a synthesis section 
36. and a 2X clock is generated by a 2X clock genera- 
tion section 37. The MPEG signal is doubled by a 2X 
clock-type MPEG decoder 1 6c. extended, and output as 
R and L video signals by a division section 38. In this 



manner, the circuit configuration can be simplified This 
circuit configuration is realized simply by adding a 16 
MB SD RAM to a memory 39 of the existing reproduc- 
tion apparatus, without significant ty raising the cost For 
f soft-decoding, when a CPU has a 2X clock, one CPU 
realizes simultaneous decoding by time division. This 
will be described in a second example. 

(Simultaneous reproduction) 

|0112] With reteience to Figure 18. synchronous 
reproduction ot two streams, which is important in 
decoding 3D picture data and progressive picture data 
will be described. First, ft is necessary to adjust vertical 

7£ and horizontal synchronization of two streams within a 
single line. In order to do this, a first MPEG decoder 16a 
and a second MPEG decoder 16b are started substan- 
tially simultaneously by a vertical/horizontal synchroni- 
zation control section 85c to synchionize the decoders 

20 1 6a and 1 6b. Then, it is necessary that the outputs from 
the two decoders should be a picture having an identical 
VPTS. This will be described with reference to the flow- 
chart in Figur e 26 and Figure 18. In step 241a. the syn- 
chronization of a first decoder and a second decoder is 

25 cancelled. In step 241 b. the decoders are synchronized 
with each other vertically and horizontally as described 
above. In step 241 c. an APTS of an audiosignal is read, 
and the APTS value is set as an initial value of an STC 
of the first decoder and an STC of the second decoder. 

3o In step 241 e, processing of the first decoder is started. 
In step 24if. it is checked whether or not a first VPTS 
has reached the initial value. If yes, decoding is started 
in step 241 g. In step 241 h, a processing delay time 
period of the first decoder is calculated, and the VPTS 

55 ol the decoder output is adjusted so that the APTS and 
the VPTS are synchronized with each other. Since the 
second decoder is processed in the same manner, the 
picture from the first decoder and the picture from the 
second decoder are synchronized with each other. 

<o Thus, the decoder outputs, i.e.. the first MPEG signal 
and the second MPEG signal are synchronized within 
one line. Then, the synchronization on a dot-by-dot 
basis is obtained by a video signal synchronization sec- 
tion 36a of the synthesis section 36. An original progr es- 

45 sive picture is obtained even by a sum calculation. As 
shown in Figure 5, in the case where an APTS 84 is 
read by the audio decoder 1 6c and an identical APTS is 
set in registers 39a and 39b of the STCs of the two 
MPEG decoders 16a and 16b, an audio stream and the 

so two video stream are automatically synchronized with 
one another. 

101 1 3] In the present invention, when the buffer cir- 
cuits 23a and 23b underflow, either one of the pictures 
is disconnected, as a result of which a disturbed pro- 
es gressive picture is output. In order to avoid this, the 
buffer amounts of the two buffer circuits are controlled 
by a buffer amount control section 23c as shown in Fig- 
ure 2. This operation is illustrated in the flowchart shown 
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in Figuie 27. First, in step 240a. a maximum interleave 
value among the NAVI irriormation ol each disk is iead. 
and a maximum value o1 1 ILB in one mam interleave 
block is set. The maximum value is usually 512 sectors, 
i.e.. about 1 MB. When the maximum value is set below 

I MB by a specific format, that value is set as the maxi- 
mum value. Next, when an instruction to simultaneously 
reproduce the main and sub interleave blocks is issued 
in step 240b. H the butter amount ot the first butlei circuit 
23a is 1 ILB oi less in step 240c. an instruction to iepro- 
duce the data fiorn the main interleave block and trans - 
fer the data to the first buffer circuit 23a is issued Then, 
the processing goes back to steps 240b and 240c. The 
transfer is stopped in step 240d when the buffer amount 
of the first buffer circuit exceeds 1 ILB. Since the data in 
the buffer circuit 23a becomes 1 ILB or more in this man 
ner, underflow is prevented. 

[0114] In step 2401. a maximum value of a sub intei- 
leave block of 1 ILB- Sub is set in the buffer circuit 23b. 
Simultaneous reproduction is performed in step 240g. 
When the data in the second buffer circuit 23b is 1/2 
ILB-Sub or less in step 240h. data is read into the buffer 
circuit in step 240j. When the data is more than 1/2 ILB- 
Sub. the reading is stopped in step 240i. 
[0115] As shown in part (4) of Figure 24. the data 
amount of 1/2 ILB is sufficient in the second buffer cir- 
cuit. Accordingly, the buffer amount can be reduced to 
half. The buffer control in Figure 27 eliminates the 
underflow of the buffer circuits, thus reducing distur- 
bance in the synthesized pictur e during reproduction. 

(Required capacity of the track buffer: Figures 23 and 
31) 

[0116] First, a method for synchronizing two video 
streams according to the present invention will be 
described. First, as shown in Figure 39, a system repro- 
duced from the optical disk is once accumulated in a 
track buffer 23 and then sent to a first video decoder 
69d and a second video decoder 69c. In the track ot the 
optical disk, a first stream A and a second stream B ol 
the progressive signal are alternately recorded on an 
interleave block-by-interleave block basis. 
[0117] First, the stream A is reproduced at 2X rota- 
tion, and data accumulation in a first track buffei 23a in 
the track buffer 23 is started. As shown in part (1 ) of Fig- 
ure 24, when t=tl to t2. data for 1 interleave block (ILB) 

II of the first video signal for 1 interleave time Ti is 
accumulated. A first track buffer data amount is 
increased, and becomes equal to 1 ILB at t=t2. Thus, 
data accumulation for i ILB of the first video signal is 
completed. At t=t2. after accumulation of data for 1 ILB 
of the first video signal corresponding to 1 GOP or more 
is completed, the second video signal (stream B) is 
reproduced from the optical disk starting from the inter- 
leave block 12. As shown in the solid line in part (4) of 
Figure 24, data accumulation of the second video signal . 
in a second track buffer 23b is started at t=t2 and con- 



tinued until t=t6. From 1=12 thiough t8. as shown in parts 
(7) and (10) of Figure 24. the video presentation time 
stamps (VPTS) of the first video signaJ and the second 
video signal are synchronized and respectively sent to 

5 the first video decoder 69c and the second video 
decoder 69d from the track butter 23a and the track 
buffer 23b. As shown in parts(8) and (n) ot Figure 24. 
the input signals are output as two pieces of video data 
after being extended by the first and second video 

to decoders 69c and 69d. The output of these pieces ol 
data starts at 1=t3. which is delayed by a video delay 
time period twd. which is required for MPEG extension 
of the data. From t=t4 through tlO. the streams A and B 
are synthesized into a progr essive signal by a pr ogr es- 

75 sive conversion section 170. Thus, a progressive signal 
for one interleave block is output. 

[0118] As described above, from t=t2 through t8. 
data for one interleave block is input to the decoders. 
Accordingly. the data in the first track buffer 23a and the 

20 data in the second track buffer 23b are consumed and 
reduced at substantially the same rate. Therefore, as 
shown in part <2) of Figure 24, the data amount in the 
first track buffer is reduced from t=t2 through t7. At t=17, 
the data amount is 1/2 of 1 ILB. Since data reproduction 

25 for the interleave block 15 starts at 1=17. the data amount 
increases until t=t8. when the data amount reaches 1 
ILB. Since data input to the first decoder 69c starts at 
1=18 as at t=t2. the data amount reduces until t=m. 
Finally, the buffer memory amount becomes 1/2 ILB. 

so [0119] With reference to part (4) of Figure 24. a 
change in the memory amount in the second track 
buffer 23a for stream B will be described. At t=t2. input 
of data Bl for the interleave block 12 of stream B in the 
second track buffer 23b starts. At the same time, trans- 

5t fer of data Bl to the second video decoder 69d starts. 
Accordingly, the buffer amount at t=t6 is 1/2 ILB. When 
2-angle recording of a progressive signal according to 
the present invention is performed, it is necessary to 
perform a track jump to the interleave block 15 over the 

ao interleave blocks 13 and 14 from time t=6 to t=7 since 
there are four streams, i.e., four interleave blocks. Dur- 
ing the jump period 197 (tj), data input from the optical 
disk is interrupted. Thus, the buffer amount of the 
stream B is reduced until 1=16, when the buffer amount 

<£ is close to zero. 

[01 20] Since input of data B2 of the interleave block 
16 starts at t=t8, the buffer amount starts increasing 
again. Al t=tn , the memory amount of the second track 
buffer is 1/2 ILB. At t=ti 1 , a track jump to the interleave 

bo block 19 of A3 over the interleave blocks 17 and 18 is per- 
formed 

[01 21 ] The above-described operation is repeated. 
[0122] Now. the minimum necessary memory 
capacity for a track buffer 23 (total capacity of the first 
55 and second track buffers 23a and 23b) according to the 
system of the present invention will be described. A 
track buffer capacity 198 indicated by the dotted line in 
part (4) of Figure 24 shows the total data amount in the 
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1irst and second track butlers 23a and 23b A conttnu- 
ous r epiodudion e> realized by setting the total capacity 
Of a rrunimum 1 1LB in the track butter 
[0123] According to the piesent invention, the total 
capacity ot the track butlers 23a and 23b is set to be 1 5 
interleave Uoo c* more tor leproduction ot h pi ogres - 
sive signal Thus overflow and underflow o1 the track 
buttei aj e p* evented 

(Method to» control the system clock) 76 

{0124) A menod to* switching the system clock 
STC between twc sti earns will be desaibed with retei- 
ence to Figute 28 A pogressrve signal includes two 
sti earns A and B ***** tx>*? streams of two interlace sig- 75 
nais fotrmng a 1 ILB piogi«^sfve signal are ietened to 
asAiandBi As shown ^ pari { i) o< Ftgure 28. data Al 
•or steam A »s reooaucec? *>g the 1/2 ILB time 
period and ail the data c «pcoo«3 n r>e butler Then, 
as shown m pan J?) o» > •ql* t 2E data *> steam B is 20 
reproduced as B " ana w*> md r> im# tx/w* ahe» Ai is 
reproduced S^nce m« <\xl* acxx ma t»am r>€» optical 
drsk is restiiaed with vtr»*rr* i- <p^? o* * »OA*e 28) as 
described above th* ?•*-> r*#»*» tirm* net overflow 
Stream A (pan (3> 0* * 7% » utm*m cioo (SCR) 25 
from the track butte* t> u» «*r^ b «. substantially in 
synchronization with tne sxa»i port j o* r** reproduction 
ot stieam E (pan (2; o< • 28 5*nce stream B is 

output at the rate ot 2X v?««m ckx> »s counted at 
the rate ot iX as shown r>p*-t *2< o» 28. i.e., at 30 

halt the rate of stream B 3u# 10 r»# tx/te- At point G, the 
stream clock is reset Tim* v p T a! which the video 
s»gnai tor stream B is output * cm the v»deo decoder 
needs to be synchronised r> ccrt^o^ ator. ol the delay 
time period "Tvd due to to ciamp** MPEG decoding 55 
time penod In thrs case at oo-w <uTi). when the 
VPTS stops rising AV iyncr* orx:atx>n control is 
restarted By checking VP lb*? o* m earn E and synchro- 
nizing VP7S1 ot stream a tD vht^» synchronization is 
realized by one-system simp* conroi VP7 Si can be <o 
used additionally 

[0125] Audio da:a o* syntonizing stieam B is 
reproduced and lUe sysit?rr uu> switched at point H 
using APTS ol stream B ai shown m part (4) o1 Figure 
28. Regarding a sub p»ctui t won* o* stream E. the STC 
can be switched in e Simoa rmnnei 
{0126] By using data 0* shoam E with priority AV 
synchronization is realized w*r> s^npio control 
[0127] Since all the data ^ streams At and A2 is 
stored in the butter memo* y m« buffer memory does not so 
overflow Stream Bi may poss£4y overflow However, 
according to the present irvpnbon tne synchronization 
control is performed using sti*am E and thus the sys- 
tem clock is switched tc contro* r*e signal tlow so that 
VPTS2 does not exceed the VP" ST threshold leve! as £f 
shown in part (6} o1 Figure 28 Theretore. the buffer 
does not overflow. 

{0126] According to the present rrvention. the audio 



signal of stieam B is used tot audio reproduction. 
Therefoie. the butlei amount of audio decoder is 
reduced to M2 Furthermore, by switching the system 
clock at point H(t=Th) as shown in part <4) ol Figure 28. 
the audio signal is reproduced smoothly without 
exceeding the APTS threshold level The sub picture 
information is also repioduced with smooth synchroni- 
zation. Accoi amply, picture, audio and sub picture (sub- 
titles 01 the like) signals are synchronized, and picture 
and audio are reproduced seamlessly with no interrup- 
tion. The audio signal and the sub picture signals of 
stream A can be omitted. 

(AV synchronization: Figures 29, 30. 31 and 33) 

[0129] AV synchronization, which is especially 
important for connection and the like when a jump is 
performed to reproduce two or three streams simultane- 
ously, will be described This is important in the present 
invention, according to which the streams of the 720P 
signal and the 480i signal, which are significantly differ- 
ent from each other in the data amount, are synchro- 
nized. 

[01 30] Figure 29 is a flowchart showing the detailed 
process of reproduction of a program chain group per- 
formed by the system control section 21 . As shown in 
Figure 29. in steps 235a. 235b and 236c. the system 
control section 21 reads corresponding program chain 
information from the volume information file or a pro- 
gram chain information table of the video file. When the 
program chain is not completed in step 23Sd. the 
processing advances to step 235e. 
[0131] In step 235e, it is determined whether or not 
the current cell and the immediately previous cell should 
be connected seamlessly referring to seamless connec- 
tion instruction information for the cell to be transferred 
next in the program chain information, rf seamless con- 
nection is necessary, the processing goes to step 2351 
tor seamless connection processing, rf not, ordinary 
connection is performed. 

[0132] In step 2351, the mechanism control section 
and the signal processing section, for example, are con- 
trolled to read DSI packets, so that VOB reproduction 
end time (VOB_E_PTM) in the DSI packet of the cell 
which has been transferred and VOB reproduction start 
time (VOB_S_PTM) in the DSI packet of the cell to be 
transferred next are read. 

[0133] In step 23Sh, "VOB reproduction end time 
(VOB_E_PTM) - VOB reproduction start time 
(VOB_S_PTV»^ is found by calculation. The resultant 
value is sent to an STC offset synthesis section 164 in 
the AV synchronization control section 158 in Figure 30 
as an STC offset value between the current cell and the 
immediately previous cell which has been transterred. 
[0134] Simultaneously, in step 235L VOB reproduc- 
tion end time (VOB_E_PTM) is transferred to an STC 
switch timing control section 166 as switching time T4 
for an STC switch I62e. 
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|0135] The system control seclion 21 then instructs 
the mechanism control section to continue reading data 
up to the terminal position ol the cunent cell. Thus, the 
data tor the cunent cell is transferred to the track buffer 
23 in step 235j. Upon completion ol the transfer, the 
program chain mtor matron is lead in step 235c. 
|Ol36] 11 it is determined the seamless connection 
is not necessary in step 235e, the data is translerred to 
the track butler 23 up to the end ol the system stream, 
and then program chain information is read in step 
235C 

10137) Hereinafter, two examples o1 a method tor 
AV synchronization control toi seamless connection to 
perl of m seamless reproduction will be described. In 
othet wtwdb. the AV synchronization contiol seclion 158 
shown m Fiyuies 2 and 31 will be described in detail. 
[0138] Re*emng to Figure 31. a system decoder 
161 an auc*c decoder 160. video decoders 69c and 
69d and a sub picture decoder 159 are all synchro- 
ny oc to a tyt.tem time clock given by the AV synchroni- 
zation comic* t^cion m Figure 30 to process the data in 
the if%.)«m vtr 

|0i39J ^M^jaf^og a first method, the AV synchioni- 
7*t>or crrm nt v^iinn 158 will be described with refer - 
por» 10 

10140] r- 30 me AV synchronization control 

sector *x^:v-* Z etches 162a. 162b. 162c and 
i6Jd ar ti3 Z1C offset synthesis section 

1M r ' " 1 v**vig ^cr. 165 and an STC swhch tim- 
ing cr:r* o yp».*ar 164 

|0i4ij * T u->*c*es 162a. 162b. 162c. I62d 
anr! i4T« vm^.r* im ^ m « m * *. an output value of the STC 
163 »nr ar +&~* o* tne STC offset synthesis 

s*ctk>~ 164 m\ « 'i^»«rx# clock to be provided to the 
svvt#*^ :**ccrv» i*i audo decoder 160. the main 
v*y*K >ca> 69c w<Jeo decoder 69d and the 

sol 3#coo*' 1S9 'effectively. 

|0i4?j t*>r it3 ^ a reference clock of the 
ens'* v*-t o >cor> vr«*n m Figure 31 in ordinary 
repoojaor 

[0143] Tn* LTC iv^mesrs section 164 contin- 

ues uu^AjTtog * .aiu* uutam^o by subtracting the STC 
ottv«v »aU u»L*vO«*j L> r»r b ^em control section from 
the *a>u* o' tr* G1C 163 

(0144] Th* G1C s^t!»r>g secnon 165 sets an STC 
muai *&\u* Q***r t>) tr* system control section <y en 
SI C oft vet syntnesrfs vatuo given by the STC offset syn- 
thetic section 1 64 m tne SI C 163 at the timing given by 
the S7C switch timing conuor section 166. 
[0145] The STC swticn timing control section 166 
controls the STC switches 162a through I62e and the 
STC setting section 165 based on STC switch timing 
iniormation given by the system control section, the 
STC 163. and the STC offset synthesis value given by 
the STC offset synthesis seclion 164. 
[0146] The STC offset value is an offset value used 
for changing the STC value when system stream #i and 
system stream #2 having different STC initial values are 



continuously reproduced. 

(0147] The STC offset value is specHicatly obtained 
by subtracting the "VOB reproduction start time 
(VOB_S_PTM)- described in the OSI ol system stream 

f #2 to be reproduced next from the "VOB reproduction 
end time ( VOB_E_PTM)~ described in the DSI packet of 
system stream #1 reproduced first. The information 
regarding the display of such a value is precalculated 
by reading data from the optical disk in Figure 5 by the 

to system contr ol section 1 67 when the data is input to the 
track butter 23. 

[01 48] The calculated offset value is supplied to the 
STC offset synthesis section 164 before the last pack of 
system stream #1 is input to the system decoder 161 . 

i5 [0149] Except for seamless connection control, the 
data decoding processing section 165 in Figure 5 oper- 
ates as an MPEG decoder. The STC offset value given 
by the system control section 21 is 0 or an arbitrary 
value. The STC switches 1 62a through 162e are always 

20 selected to be connected to the STC 163. 

[01 50] With reference to the flowchart in Figure 33, 
switching of the STC switches 162a through 162e in the 
connection part of the system control section and an 
operation of the STC 163 when two system streams 

pa having non-continuous STC values, such as system 
streams #1 and #2, are continuously input to the system 
decoder 161 . will be described. 

[0151] The SCR. APTS. VPTS and VDTS of the 
system streams #T and #2 to be input will not be 

so described. 

[01 52] It is assumed that in the STC 1 63, an Initial 
STC value corresponding to system stream #1 which is 
being reproduced is set by the STC setting section 165. 
and the value is sequentially counted up in accordance 

3f with the reproduction. The system control section 21 
(Figure 31) calculates the STC offset value by the 
above-described method and sets this value in the STC 
offset synthesis section 164 before the last pack of sys- 
tem stream *n is input to the decoder buffer. The STC 

<o offset synthesis section 164 continues outputting a 
value obtained by subtracting the STC offset value from 
the value of the STC 163 (step 168a). 
[0153] The STC switch timing control section 166 
obtains time Ti , at which the last pack of system stream 

<i 4 £1 reproduced first is input to the decoder buffer, and 
switches the STC switch 162a to the output side of the 
STC offset synthesis section 164 at time Ti (step 
168b) 

[01 54] Thereafter, the STC value referred to by the 
so system decoder 1 61 is provided with an output from the 
STC offset synthesis section 164. The transfer timing of 
system stream #2 to the system decoder 161 is deter- 
mined by the SCR described in the pack header of sys- 
tem stream #2. 
£5 [01 55] Next, the STC switch timing control section 
166 obtains time T2, at which the reproduction of the 
last audio frame of system stream #1 reproduced first is 
terminated, and switches the STC switch 162b to the 
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output side o1 STC offset synthesis section 164 al 
time T2 {step "J bSc). A method for obtaining time 12 wil 
be described laiet. 

f01S6] Thereafter, the STC value retened to by the 
audio decoder 160 is provided with an output liom the 
STC offset synthesis section 164. The audio output tim 
ing ol system stream £2 is detei mined by Ihe APTS 
described in the audio packet of system stream $.2. 
(0157] Next, the STC switch timing control section 
T66 obtains time T3 and T3\ at which the decoding ol 
the last video frame ol the mam signal and the sub sig- 
nal of system stream #1 reproduced first is terminated, 
and switches the STC switches 162c and I62d to the 
oufc)Ut side of the STC offset synthesis section 1 64 at 
time T3 and T3' (step 168d) A method foi obtaining 
time T3 will be described later. Theieaftei, the STC 
value referred to by the video decoders 69c and 69d is 
provided with an output from the STC offset synthesis 
section 164. The video decoding timing of system 
stream #2 is determined by the VPTS described in the 
video packet of system stream #2. 
[0158] Next, the STC switch timing control section 
166 obtains time T4, at which the reproduction output of 
the last video frame of system stream #1 reproduced 
first is terminated, and switches the STC switch I62e to 
the output side of the STC offset synthesis section 164 
at time T4 (step 168e). A method for obtaining time T4 
will be described later. 

[01S9] Thereafter, the STC value referred to by the 
video output switch 169 and the sub picture decoder 
159 is provided with an output from the STC offset syn- 
thesis section 164. The video output timing and sub pic- 
ture output timing of system stream #2 are determined 
by the VPTS and SPTS described in the video packet 
and the sub picture packet of system stream #2. 
(0160] When switching of the STC switches 162a 
through r62e is completed, the STC setting section 165 
sets the value given by the STC offset synthesis section 
164 in the STC 162 (step I68f) (referred to as "reload- 
ing of the STC 163) and switches all the switches 162a 
through I62e to be connected to the STC 163 (step 
16Bg). 

(0161] Thereafter, the STC value referred to by the 
audio decoder 160. the video decoders 69c and 69d. 
the video output switch 169 and the sub picture decoder 
159 is provided with an output from the STC 163, and 
the operation returns to the ordinary operation. 
{0162] Now. two means for obtaining time Tl 
through T4 for switching the STC will be described. 
{0163] According to specific means, information 
representing time Tl through T4. which can be easily 
calculated when the streams are created, is recorded 
on the disk The system control section 21 reads the 
information and sends the information to the STC switch 
timing control section 166. 

[0164] Especially for 14, "VOB reproduction end 
time (VOB_E_PTMr described in the DSI used for 
obtaining the STC offset is used as it is. 



|01 65] On the disk, the value obtained based on ihe 
STC value used in system stream *i reproduced tiist is 
described, and the STC switch timing control section 
166 switches the STC switches 162a through I62e at 
the moment the value of the STC 163 becomes time Tl 
through 14. 

(Example 2) 

7r [0166] In the first example, an example of applica- 
tion ot a system tor reproducing a pluralrty ol streams in 
synchronization according to the present invention is 
described in detail. In a second example, this system is 
applied to a reproduction control system for reproducing 

it two streams seamlessly In the case of recording an 
MPEG signal, editing is conventionally performed on a 
GOP-by-GOP basis in general, and it is difficult by con- 
ventional methods to perform editing on a frame-by- 
frame basis. By using an MSS system according to the 

pc present invention, substantial frame-based editing is 
realized. 

[0167] It is important to synchronize the timing of 
the video signal and the audio signal at the point of con- 
nection. Specific synchronization systems will be 

25 described in third through ninth examples. 

[0168] The reproduction control system according 
to the present invention can be applied as follows to 
connect two streams while switching the two streams on 
a frame-by-frame basis seamlessly. As shown in Figure 

so 1 0, editing data 761 including synthesis information of a 
28P zoom instruction signal and the like is processed by 
an editing data processing section 762 and sent to a 
switching synthesis section 763. The data is then 
switched/synthesized and output from a switching syn- 

££ thesis signal output section 764. Thus, two MPEG video 
signals can be switched and connected seamlessly at 
an arbitrary point other than the borders between 
GOPs. 

[0169] A system for synthesizing two pictures at an 
46 arbitrary point based on an instruction signal will be 
described later with reference to Figure 22. 
[0170] In a simple switching mode, two pictures are 
simply switched over frame by frame at an editing point 
as follows. Stream "a" and stream "b" are switched over 
<5 at an editing point tc. and the resultant stream is output 
seamlessly. In a synthesis switching mode <such as a 
wipe), stream "e r and stream "b* are switched after 
being synthesized from a start point ts to a termination 
point te. As shown in Figures 6 and 58. in mode 1, 
so switching is performed from left to right; in mode 2, from 
center to periphery; in mode S. top to bottom; and in 
mode 4. in a mosaic manner. Figure 6 is a simplified 
block diagram; and Figure 58 is a detailed block dia- 
gram. 

5f [0171] In Figure 6, reproduction means 778. a divi- 
sion section 734. a VTS synchronization section 780. 
and MPEG decoders 728 and 730 have exactly the 
same structure as that of those in the 480P reproduc- 
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tion appaiatus shown in Figure 3. and thus Ihe 480P 
reproduction apparatus can be used. With leieience to 
Figure 6. when an identification infoimation processing 
section 766 detects reproduction control inloimation 
766a. two video streams aie sent to the switching syn- b 
thesis section 763. and the tirst stream is switched to 
the second stieam seamlessly at the connection point ts 
as desciibed above. 

{0172] With reference to Figuie 58. when a 
720P/480P hierarchical recording identifier 725 having m 
a high lesolution signal described in the first example is 
detected, the synthesis section 732a periorms calcula- 
tion and outputs a high resolution signal such as a 480P 
or 720P signal. 

[0173] When a 3D recoiding idenlrliei 766c is 15 
detected a 3D signal processing section 770 generates 
a 3D video signal having a right-eye video signal and a 
lett-eye video signal alternately interleaved, and outputs 
the signal. 

[0174] In this manner, the MSS system shown in 20 
Figure 58. which uses two MPEG decoders or an 
MPEG decoder for decoding two streams simultane- 
ously, can be used to provide three lunctions. i.e.. repro- 
duction control with frame-based editing, reproduction 
of a high resolution signal, and reproduction of a 3D 25 
video signal. 

[0175] Figure 1 1 shows a specific example of repro- 
duction control information 765. The reproduction con- 
trol information 765 includes a switching point number S 
766. a synthesis mode identifier 767. a first stream 
switching start address ts 768. a first stieam switching 
termination address te2 769. a second stream GOP 
start address tsG 790. a second stream switching start 
address ts2 771 . and a second stream switching termi- 
nation address te2. 772 in the order of the address 
used. 

[0176] Specifically, when the switching point 
number S=i. as shown in part (9) of Figure 12, a picture 
synthesis identifier 767 is not present or is 0. Accord- 
ingly, the tirst stream is simply switched into the second 
stream at the switching address of tsi-1. When S=2. as 
shown in part (10) of Figure 12. switching is started at 
tsi. and the two pictures of the first stream and the sec- 
ond stream are synthesized into one picture until t=fei . 
At t=tei . the first stream is completely switched into the 
second stream. 

[0177] With reference to the flowchart of Figure 13. 
a process for reproduction performed based on the 
reproduction control information will be described. 
[0178] With reference to part (1) of Figure 12, a 
GOP 781a of the first stream and a GOP 781b of the 
second stream may be at discrete positions on an opti- 
cal disk without the streams being moved or rewritten. In 
this case, the time period for rewriting can be saved. 
With reference to part (3) of Figure 12. since data can 
be recorded on a DVD- RAM disk and the like after 
edited, streams are recorded one GOP-by-GOP basis; 
i.e.. a GOP 781 e and a GOP 781 f both including a 



tiame at the editing point aie lecoided adjacent to each 
othei. In this case, the editing point is allowed to be 
changed in one GOP latei. When two streams aie syn- 
thesized in a wipe manner, the pictuie of the first stieam 
attei the editing point is lequired. Thus, the structure 
shown in part (3) ot Figure 12 is lequiied. 
[0179] When S=0. i.e.. when synthesis of two pic- 
tures is not performed, the data after the switching point 
tsi of a GOP 781 c is not required and thus is deleted as 
shown in part (2) of Figur e 12 to exclude the redundant 
portion. Thus, the recording efficiency is raised. How- 
ever, a GOP 78id. which has the IN point, includes an i 
(intra) frame (i.e.. basic frame) at the beginning, which 
cannot be deleted. Thus, a ledundant portion 783 is 
generated. 

[0180] As shown in step 792a in Figuie 59. 1GOP 
includes about 15 actual frames. When the B frames 
present before the I N point are deleted as in step 792b. 
the number of frames is reduced from 12 in step 792a 
(redundant portion 783f) to 3 in step 792c (redundant 
portion 783g). The redundant portion 783g is about 1/4 
of the ledundant portion 7831. and thus the recording 
efficiency is raised. 

(0181] When this portion is reproduced, a 6 frame 
deletion identifier is detected in step 792f. The number 
of frames are calculated with a premise that the B 
frames are not recorded. Since an MPEG signal is 
decoded with only I. P frames, the frames are decoded 
one after another in step 792g. The frame having the IN 
point (i.e.. t=ts2 ) is obtained by decoding and output. In 
this case, the number of frames to be processed is only 
three. Accordingly, the intended IN point can be repro- 
duced in e 1/4 time period. In this example, the redun- 
dant portion is 1/8 second long. As can be appreciated 
from Figure 59. ts2 is in the 14th frame in the worst 
case. In this case, the redundant portion is five frames 
long of I, P, P. P, B, i.e., 1/3 * 1/2 = 1/6 second long. That 
is, the longest possible redundant portion is about 0.18 
second. At least such a time period is necessary to 
reproduce the IN point. Even if about five cut portions 
are included in one second, each cut portion is present 
every 0.2 second. By deleting the B frames by this sys- 
tem, the frame-based reproduction is realized even if 
there are five cut portions in one second. This means 
that this system is usable in the standard editing with no 
problem. 

[0182] A method for generating reproduction con- 
trol information will be described. Where the final GOP 
before the OUT point is defined as a first GOP and the 
first GOP after the IN point is defined as a second GOP. 
reproduction control can be performed simply by 
recording the time of the beginning of the second GOP 
as the switching point ts2 and the lime of the switching 
point. Alternatively, the number of frames from the 
beginning of the second GOP to the switching point can 
be recorded. 

[0183] When such reproduction control information 
is reproduced by a reproduction apparatus, the frame at 
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the switching point is decoded without piocessmg the B 
frames (pictures) among I. B. E. B. P. B. B. B. P. B. B. B 
as shown in step 792f in Figuie 59. In othei woids. only 
I. P. P aie decoded In this mannei. the piciuie at the IN 
point is obtained in a i/< time penod as described 
above Even it there aie live frame switching points in 
one second, the redundant time penod is 0.18 second, 
which is shorter than the pitch ot the cut portion (0 ? 
second) Thus, the data can be reproduced seamlessly 
at all the switching points. 

{0184] In order to lealize synchronization, repro- 
duction control intonation is obtained by calculating the 
number ot tiames (pictures) existing between the sec- 
ond OOP and the switching point based on the time ot 
the beginning of the second GOP and the time ol the 
switching point When the B fiames which aie not nec- 
essary aie deleted, the correction is performed in con- 
sideration of the deleted -frames referring to an 
unnecessary tiamc deletion identifier. Then, it can be 
found ho* man> fcamoc should exist between the start 
ol reproduction of tn# tecond GOP, the IN point ol the 
second OOP and tr* OUT point ol the first GOP, in 
ordei to vy»ic^oncr* OUT point of the first GOP and 
the IN prvn ntf rt-w w^/yn GOP 

[0185) *f\^w^ tr*. r+mttet ol frames existing 
betwe**~- trw be^nr*og o* tne second GOF and the 
swrtctwg port ^ tm&aco as the reproduction control 
inJorrna%orr co'^rbon ^ consideration of the unneces- 
sary fc*<~*% vucr ft N E fcames can be performed in 
orde« tc '*tt p^p tr^r t u*t oecoding the second GOP 
to real,-* tr*r v r* orv.- #&ar 

{0186] Artcr^m '*carCdt>e reproduction control 
information car- o» Dwcocfcog start timing information 
whicf" n>cj?ei * vcmct-a pomt in the first GOP at 
whic** o>caovis a* u#cona 3CP should be sterted in 
ordet tc mate* w^trwxj pomt of the first GOP and 
the swtcr*ng por* o* v*cond GOP 
(01 87] r-, VLicf*. a cav* V* switching points can be 
syncr* arista uyng om, r* tep*oduction control infor- 
mation •wrnoui perx^rr^ any special calculation by 
the repioajcbon apparatus* 

(0188] ""h* i«dLrni*nc portion 7B3 includes one I 
fram* « piu^M> of P ti«rT*s* «r ad a pluiality of B frames 
8y decoding tn*-*- t arrns, to oeate a frame immedi- 
ately oe*o»e the frame corresponding the final edrting 
point (i e . irrtratia-ne) r*e recording efficiency can fur- 
ther be raised In the case of a DVD- RAM disk the over- 
all editing siructme is found as toe by recording the 
overall reproduction control information 765 and limited 
reproduction control intormation 765a on only the 
switching points, at two porrts. i e . at the beginning of 
the recording data and immediately before the editing 
point as shown in parts (1). (2) and (3) of Figure 12 
Before the editing point, limited reproduction control 
information 765a on each separate editing point, for 
example. S= 1 can be recordec among the overall repro- 
duction control information In this case, reproduction 
control for special reproduction is advantageously stabi- 



lized 

(0189] A piocess for reproduction control will be 
described Fust, in step 774a. reproduction control infor- 
mation is lead In step 774b, the reproduction switching 

5 point numbei S is set to 0. In step 774c, S is incre- 
mented by one The decoding start position of the sec- 
ond stieam needs to be specified. In step 774d, it is 
checked whether or not t=ta . where t is the system 
dock or the VPTS of the first stream and ta is the decod- 

io ing start position information. When t=ta . i.e.. when the 
VPTS ot the second stream reaches ta as shown in part 
(5) of Figure 12. the processing advances to step 774e. 
wheie MPEG-decoding of the GOP of the second 
stream is started as shown in part (6) of Figure 12. In 

75 step 774 f. it is checked whethei oi not t=ts1 . As shown 
in parts (5) and (6) of Figure 12, the switching point ts2 
comes attei the time corresponding to the value of 
(tsl-ta)=(ts2-tsG) passes. As shown by the expression 
and parts (5) and (6) of Figure 12, the data at tsl of the 

20 first stream and data at ts2 of the second stream are 
MPEG-decoded at the same time and output from the 
decoder. Parts (8) and (9) of Figure 12 show the state 
where the two streams are synchronized. Specific 
methods for synchronization will be described in detail 

25 in the third through ninth examples. It is appreciated 
from the reproduction control information that tsl and 
ts2 are frame editing points o1 the two pictures. In step 
774g. it is determined whether or not there is a picture 
synthesis identrfier 767. If the picture synthesis identifier 

30 767 does not exist, the processing advances to step 
774h, where the first stream is switched into the second 
stream by the switching synthesis section 763 (Figure 6) 
at the position of t=tsl as shown in part (10) of Figure 
12. Since 1=tsi is the OUT point in actuality, the frame 

35 o1 the first stream at t=ts1 is not output. Since t=ts2 is 
the IN point, the frame of the second stream at this point 
is output Since the frame information on the first stream 
is not necessary at t=ts2 , recording of such information 
can be omrrtec when S=0, i.e., when no picture synthe- 

<o sis is performed In this case, the recording efficiency is 
raised by one F frame. Thus, reproduction control of the 
switching point for S=l (simple switching mode) is com- 
pleted The processing returns to step 774c and deals 
with the switching point of S=2. 

<5 [0190] If n=0 in step 774R it is converted into n=l. 
In other words, when the second stream is decoded by 
the second MPEG decoder 730, the second stream, 
i.e., the MPEG signal to be switched next is decoded by 
the first MPEG decoder 728. A different MPEG decoder 

so is used from the case of S=1. 

[01 91 ] The second stream is decoded as shown in 
Figure 56 Input second streams 781 a, 781b, 781g, 
781 h and 781 i are sent by the division section 734 to 
the first MPEG decoder 728 and the second MP€G 

55 decoder 730 alternately and decoded into video signals 
788a and 788b The video signals 788a and 788b are 
synthesized into one stream by the switching section 
763. As is clear from the figure, the output from the first 
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MPEG decoder 728 is slopped atlei the decoded video 
signal 788a is output. That is. the pictuie is liozen. This 
occuis since non- continuous data cannot be normally 
decoded. The MPEG decoding is resumed by perform- 
ing resetting 790 ol registers and the like The second 
MPEG decodei is piocessed similarly. Conventionally, 
since only one MPEG decodei is used to perioi m seam- 
less reproduction, various complicated pre-processing 
needs to be pertoimed at the time ot recording. Accord- 
ing to the present invention, such various complicated 
piocessing is not necessary at the time ol lecoiding lor 
the tollowing reason. Since even il one ol the MPEG 
decoders is stopped, the other MPEG decodei can be 
used, even MPEG data which could not otherwise be 
connected seamlessly can be reproduced seamlessly. 
While the second MPEG decoder operates, the first 
MPEG decoder is processed to resume operating. In 
this manner, seamless reproduction can be continued 
endlessly by switching the two MPEG decoders. Even 
trame-based editing o1 MPEG signals, which conven- 
tionally requires complicated processing, can be real- 
ized without complicated processing by using 
reproduction control according to the present invention. 
The system according to the present invention does not 
require the step o1 decoding and re-encoding MPEG 
data and thus provides a significant effect that the pic- 
ture quality is not deteriorated. 

(0192] The process tor seamless reproduction can 
be summarized as follows. A plurality of streams are 
divided and alternately decoded by two MPEG decod- 
ers. An output from one of the MPEG decoders is 
switched into an output from the other MPEG decoder 
at a switching point. While the other MPEG decoder is 
outputting data, the first MPEG decoder is reset to 
decode the next stream. At the next switching point, the 
output from the second MPEG decoder is switched into 
the output from the first MPEG decoder. 
{0193] Thus, continuous trame-based reproduction 
is realized. 

(Synthesis switching mode) 

(0194] Returning to step 774g of Figure 13, a proc- 
ess for switching two streams while synthesizing the two 
streams into one picture in for example, a wipe manner, 
by the synthesis signal output section 764 (shown in 
Figure 6) will be described. In this case, the picture syn- 
thesis identifier 767 is not 0. In step 774i, the first 
stream is switched into the second stream as shown in 
part (11) of Figure 12 while the two streams are synthe- 
sized from t=tsl . The switching is continued until the 
time becomes t=te1 or t=te2 in step 774j. The switch- 
ing is completed in step 774k. Since the decoding of the 
first stream is stopped at t=tsi , wasteful decoding of 
data is prevented. The switching is performed from left 
to right as in the screen 782 a in mode 1 , from center to 
periphery as in the screen 782b in mode 2, top to bot- 
tom as in the screen 782c in mode 3, and in a mosaic 



manner as in the screen 782d in mode 4. 
(0195] Figuie 12 shows an example in which the 
original time stamp is not changed. When the recording 
is performed using a DVD- RAM disk or the like where 

5 the time ol the switching point of the first stream 
matches the time of the switching point ol the second 
stream, the time stamp can be changed and thus the 
structure is turthei simplified. The time stamp <»n be 
changed at the time of recording by setting ts1=ts2. In 

to this case, the lirst stream can be recorded up to tel. 
Time stamp tsG ot the second stream is changed to 
tsi -(ts2-tsG) . A time stamp having a smaller value than 
tsl is assigned. Time ta is identical with tsG. Accord- 
ingly, when reproduction is performed as shown in part 

75 (6) ot Figure 12, tsG, i.e., the GOP start address 770a 
of the second stream is pie-read based on the repro- 
duction control information 765b shown in Figure 14, 
and decoding is stated at tsG. 

(0196] When a system of changing the time stamp 
20 is used, the time stamp of a GOP 761 e and the time 
stamp of a GOP 781 f in part (3) of Figure 12 are 
changed in the order and also overlapped with each 
other. When such a stream is reproduced, the reproduc- 
tion apparatus malfunctions According to the present 
25 invention as shown in Figure 14, the-GOP start address 
770a of the second stream is recorded in the reproduc- 
tion control information 765b. Thus, the positions of the 
time stamps which are changed in the order and also 
overlapped are found in advance. Therefore, malfunc- 
3& tion can be advantageously prevented even while fast- 
forwarding. Use of the GOP start address 770a further 
secures the switching into the second stream at the 
editing point. 

(0197] When the reproduction is performed as 
35 shown in part (6) ol Figure 12, tsG. i.e., the GOP lead- 
ing address 770a of the second stream is pre-read 
based on the reproduction control information shown in 
Figure 14, and decoding is started at tsG. 
(01 98] Alternatively, when information on the editing 
ao point tsl is available, the leading address tsG of the 
GOP including the editing point is pre-read and the data 
on the GOP 781 f of the second stream is MPEG- 
decoded at t=tsG . This provides an advantage that the 
editing point of the first steam and the editing point of 
<5 the second stream automatically match at tsl . 

[0199] By recording data with the time stamps 
being changed as above, the structure and operation 
can be significantly simplified. 

(0200] The above-mentioned second recording 
so method will be described with reference to Figures 15 
and 16. 

(0201] With reference to Figures 6 and 16, an edit- 
ing/reproduction control information generation pro- 
gram will be described. As shown in Figure 6. editing 
55 information 780 including the IN and OUT points (edit- 
ing points) is input manually or based on data and 
sequentially converted into reproduction control infor- 
mation 765 by a reproduction control information gener- 
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ation section 789. The reproduction control information 
765 is once stored in a memory 779 and recorded by 
recording means 777 erf the RAM disk 724 tiom the 
memory 779 either after all the editing operation is com 
pleted oi immediately betore the disk is removed tiom 
the apparatus. 

(0202] A process performed by the tepioduclion 
control information generation section 789 will be 
described in detail with reference to Figure 16 
[0203] In step 785a. the editing information 780 is 
input manually or based on data sequentially S repre- 
sents the number of the editing point, and G represents 
modes 1 through A in which two pictures aie switched 
while being synthesized by wipe or the like. tsO repre- 
sents the start point of the first stieam. tsi repiesenis 
the OUT point of the first stieam, lei represents the 
OUT synthesis completion point of the first stieem. ts2 
represents is the IN point oi the second stream. te2 iep- 
resents the IN synthesis completion point of the second 
stream, and tL2 is the OUT point of the second stream. 
10204] fn step 785b. S=0 is set. S is incremented by 
one (step 785c). and tsO and tsi are read (step 78 5d). 
In step 78Se. it is checked whether or not G exists, tf G 
does not exist, there is no need to synthesize pictures, 
and thus in step 785f, the first stream (S) is r ecorded on 
the optical disk 724 from tsO to tsi. The IN point ts2 of 
the second stream (S+i) is read, and the processing 
goes into a time stamp conversion routine (step 785g). 
In the time stamp conversion routine, the leading time 
stamp tsG of the leading GOP including inlormation cor- 
responding to the frame conesponding to ts2, and the 
time stamp ts2 corresponding to the final frame among 
the frames corresponding to the leading GOP are 
obtained. The value oi the time stamp is reduced by the 
amount conesponding to (ts2-tsl) from tsG to tL2 of al! 
the recording data of the second stream, thereby gener- 
ating a new time stamp. In step 785i, original address 
tsG to tl_2 <tt2) of the second stream is replaced with the 
new time stamp, and the information conesponding to 
the frame immediately following the frame correspond- 
ing to tsi (OUT point of the first stream), is rewritten by 
the information obtained by the new time stamp. 
{0205] In step 785w. the address tsG of the leading 
GOP of the second stieam is used to perform the calcu- 
lation of ta=tsl-(ts2-tsG} so as to find the time period te 
prior to the decoding of the second stream. The result- 
ant time period is added to the limited reproduction con- 
trol information 765a and the reproduction control 
information 765. In step 785j. limited reproduction con- 
trol information 765a only on the S number is recorded 
in the second half of the first stream (S) as shown in part 
(2) of Figure 12. H S is terminated in step 785j, the 
processing advances to step 785m; and if S is not ter- 
minated in step 785j. the processing returns to step 
785c to repeat the steps. In step 785m, the reproduc- 
tion control information 765 at all the editing points 
stored in the memory 779 is recorded in the reproduc- 
tion control information recording section of the optical 



disk wheie management information such as TOC and 
the like are i ecorded. as shown in part (3) o1 Figure 12 
Thus, the editing/iepioduction control information gen- 
eration program is completed 

f [0206] If a synthesis identifier G exists, in step 
785n. tel (OUT synthesis completion point of the first 
stream) is read. In step 785p. the calculation of 
tG=tel -tsi is performed, ff tG<rtGmax in step 7S5q. the 
processing advances to step 785r. If tG>tGmax. the 

n> synthesis time period oi the two streams is very long 
and exceeds the capacity o1 the buffer oi the reproduc- 
tion apparatus. In step 785 v. an error message that 
"deciease teV is issued. When tel is changed in step 
785 w, the processing returns to step 785n. and tel is 

75. decreased to be within the capacity. 

[0207] Thus, the synthesis of the connection point 
is within the capacity. Accordingly, the processing 
returns to step 785r. where the first stream is recorded 
on the optical disk fiom tsG to tel . In step 785s, ts2 of 

po the second stream is read; and in step 785t, the time 
stamp conversion processing routine is performed in a 
similar manner to the step 785g. to convert the time 
stamp, tn step 785u, the original address tsG to tf2 (tL2) 
of the second stream is replaced with the new time 

pa stamp, and the information corresponding to the frame 
immediately following the frame corresponding tel of 
the f r>st stream is rewritten with the information obtained 
by the new stamp. Time ta is recorded in the memory. In 
step 785 j. the limited reproduction control information 

so 765a is recorded. In step 785k, it is checked whether or 
not processing for S=1 through S=n is completed. In 
step 785m. the overall reproduction control information 
765 is recorded on the optical disk. Thus, all the opera- 
tions are completed. 

se [0208] In this manner, the first stream is recorded 
up to a connection point, and a portion after the connec- 
tion point is rewritten with a time stamp having a smaller 
value of that of the connection point, so that the connec- 
tion of the first stream and the connection point ot the 

<o second stream match each other. Thus, the reproduc- 
tion apparatus can output a video signal including the 
two streams connected to each other on a trarne-by- 
frame basis. 

[0209] In this case, two MPEG decoders are 
<e required. The recording and reproduction apparatus 
shown in Figure 6 includes the MPEG encoder 791 for 
MPEG-encoding the input video signal. Since an MPEG 
encoder is never used at the same time as an MPEG 
decoder and also the MPEG encoder has a processing 
50 capability of twice or more of the MPE<3 decoder, one 
MPEG processing section has a function corresponding 
to one encoder or two decoders. 

[0210] Accordingly, when the present invention is 
applied to the recording and reproduction apparatus 
55 including an MPEG encoder, the frame-based editing is 
realized without adding any element. 
[0211] Soft-encoding/decoding performed by a 
CPU will be described. The capability of a CPU for 
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encoding one stream corresponds to ihe capability foi 
decoding two streams. As shown in Figure 57. two 
streams can be decoded simultaneously or in a time 
division manner. Thus, the virtual fiame-based editing 
accoiding to the present invention is realized without 
raising the processing capability ot the CPU. 
|0212] With reference to the flowchart ol Figure 57. 
a process tor performing encoding/decoding and 
i ecording/i eproduction using a CPU will be described. 
In step 792a of encode recording, data ol m=l to final is 
input. In step 792c and 792d. the m'th video signal is 
input. The mth video signal is encoded to create an mlh 
stieam (step 792e) and recorded on an optical disk 
(step 7921). If m is not final in step 792g. the processing 
teluins to step 792c; if m is final in step 792g. the 
lecoiding is terminated (step 792h). In step 792i of 
recording reproduction control information, editing is 
performed on a frame-by -frame basis, and the repro- 
duction control information and the above-mentioned 
various identifiers are recorded on the optical disk (Fig- 
ures 6 and 58). 

[0213] Reproduction is performed as lollows. The 
reproduction control program 792 j is started. Cut point 
S is reproduced from 1 to final (steps 774b and 774c). 
As shown in steps 774m and 774e, two streams are 
MPEG<fecoded simultaneously or in a time division 
manner at the 1rame-edited point. In step 774h. one 
decoded stream is switched to the other decoded 
stream at t=ts . This operation is repeated until the final 
S is processed in step 774r. 

[0214] The CPU has a capabilrty of MPEG-encod- 
ing one stream. This means the CPU has a capability of 
MPEG-decoding two or three streams. Accordingly, one 
CPU can MPEG-encode one stream, performs frame- 
based editing. MPEG-decodes two streams, and out- 
puts the connected stream seamlessly. The present 
invention has an eflect that a part of the capability of the 
CPU which is not conventionally used is effectively 
used. 

(Example 3) 

[0215] The MADM system according to the present 
invention simultaneously ieproduce a plurality of 
streams. Synchronization systems are important. 
[0216] In the first example, recording and reproduc- 
tion of high resolution video signals such as 480P and 
720F signals are described. In the second example, 
basic AV synchronization systems for reproduction con- 
trol using virtual frame-based editing are described. In 
the third through ninth examples, various methods of 
synchronization will be described in more detail. 
[0217] In the third example, en operation of a repro- 
duction apparatus for reading data from an optical disk 
having three compression video signals to be repro- 
duced simultaneously, and extending and reproducing 
the three compression video signals simultaneously by 
an AV synchronization system will be described. 



[0218] Figur e 37 shows a data structure of an opti- 
cal disk used in the optical disk reproduction apparatus 
in the third example. 

[0219] Video signals A. B and C are MPEG-com- 
pressed to obtain compression video streams A. B and 

C. 

[0220] The compression video streams A. B and C 
are each packet ed in units of 2 kB into video packets. A 
packet header of each packet includes a stream ID for 

to indicating which one of the compression video streams 
A through C is stored. When the packet stores a leading 
part ot the video frame, the packet header also includes 
a VPTS (video presentation time stamp) as video repro- 
duction time information indicating the time to repro- 

75 duce the fiame. In the third example, an NTSC signal is 
used as the video signal, and the video frame cycle is 
about 33 msec. 

[0221] On the optical disk, video packets created in 
the above -described manner are grouped into, for 

20 example, compression video signals A-1, B-1 and C-1 
each including an appropriate number of packets based 
on the data stored, and multiplexed. 
[0222] Figure 35 is a block diagram of an optical 
disk repr oduction apparatus in the third example. 

25 [0223] In Figure 35. the optical disk reproduction 
apparatus includes an optical disk 501 described above, 
an optical pickup 502 for reading data from the optical 
disk 501 . signal processing means 503 for performing a 
series of signal processing such as binarization, 

so demodulation, and error correction to the signal read by 
the optical pickup 502, a buffer memory 504 for tempo- 
rarily storing the data output from the signal processing 
means 503, division means 505 for dividing the data 
read from the buffer memory 504 into compression 

5$ video, signals, and reference time signal generation 
means 506 for generating a reference time signal 506 
including a counter (not shown) for counting 90 kHz 
clocks. Reference numerals 510. 520 and 530 repre- 
sent butter memories tor temporarily storing the com- 

40 pression video signals divided by the division means 
505. Reference numerals 511, 521 and 531 represent 
video decoders for extending and reproducing the com- 
pression video signals. Reference numerals 512, 522 
and 532 represent monitors for displaying the video sig- 

*e nals. 

[0224] Figure 36 shows the structure of each of the 
video decoders 51 1 , 521 and 531 . 
[0225] As shown in Figure 36, the video decoder 
includes VPTS detection means 601 for detecting a 

so VPTS stored in the packet header of the video packet, 
video extension means 602 for MPEG-extending the 
compression video stream, and video reproduction tim- 
ing control means 603 for comparing the reference time 
signal and the VPTS and skipping or repeating the 

£f video reproduction on a frame-by -frame basis when the 
comparison result exceeds the threshold value. 
[0226] The optical disk reproduction apparatus 
shown in Figure 35 operates in the following manner. 
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[0227] The optical pickup 502 is locus- cont rolled oi 
tracking-conl rolled by servo means (not shown) to read 
a signal from the optical disk 501 and outputs the signai 
to the signal processing means 503. The signa 
processing means 503 subjects the signal to a set ies o* 
processings including binari2ation, demodulation, enoi 
conection and the like. Then, the signa! processing 
means 503 stores the resultant signal in the buttei 
memory 504 as digital data. 

{0228] The buffer memory 504 functions so that 
even when the data supply trom the optical disk 501 is 
temporarily stopped by. e.g.. a wait state, the data sup 
ply to the subsequent-stage sections is not stopped. 
{0229] The data read from the buffer memory 504 is 
divided into compression video signals A thiough C by 
the division means 505 and output. The division means 
identifies which of the compression video streams A 
through C is stored in each packet using the stream ID 
in the packet header of the packet ed data, and deter 
mines the destination based on the identification result 
{0230] The divided compression video signals are 
respectively stored in buffer memories 51 0 through 530 
{0231] The buffer memories 51 0 through 530 act to 
continuously supply data to the video decoders 511 
through S3! . 

{0232] The video decoders 511 through 531 read 
data from the buffer memories 510 through 530 respec- 
tively, extend the compression video signals, and output 
the signals as video signals to the monitors 512 through 
532 respectively. 

{0233] With reference to Figure 36. operation of the 
video decoders 511 through 531 will be described. 
{0234] The compression video signal read from the 
buffer memory is input to the VPTS detection means 
601 and the video extension means 602. 
{0235] The video extension means 602 MPEG- 
extends the compression video stream and outputs the 
video signal. 

[0236] The VPTS detection means 601 detects the 
VPTS of the packet header and outputs the VPTS. 
{0237] The video reproduction timing control means 
603 receives the video signal output from the video 
extension means 602. a relerence time signal and the 
VPTS output from the VPTS detection means 601 , and 
compares the reference time signal and the VPTS 
Then the difference between the two exceeds the 
threshold value, the video reproduction timing is control- 
led so that the difference between the VPTS and the ref 
erence time signal is equal to or less than the threshold 
value. 

[0238] In the third example. 33 msec is used as the 
threshold value for video reproduction. The video repro- 
duction timing control means 603 performs the follow- 
ing. 

(reference time signal-VPTS)>33 msec.:! frame is 
skipped. 

(reference time signal-VFTS)<-S3 msec.rl frame is 



repeated 

[0239] In the third example, due to the precision 
enoi of the crystal oscillator used in the lelerence time 
signal generation means 506 and the video decoders 
511 thiough 531. the video decoders 511 and 531 aie 
slowei and the video decoder 521 is faster in terms o1 
extension and reproduction relative to the reterence 
time signal. Unless reproduction timing is adjusted, the 

k reproduced video signals are out oi synchronization. 
[0240] Figure 36 is a timing diagram of video repio- 
duction in the third example. Part (a) of Figur e 38 shows 
the reference time signal with respect to reproduction 
time t. Pari (b) shows the VPTS* A. which is a VPTS of 

75 the compression video signal A to be extended by the 
video decoder 51 1 , part (c) shows the VPTS#6, which 
is a VPTS oi the compression video signal B to be 
extended by the video decoder 521 , and part (d) shows 
the VPTS#C. which is a VPTS of the compression video 

pc signal C to be extended by the video decoder 531 . 

[0241] The video decoder 511 continues extension 
and reproduction of the compression video signal A, 
and the difference between the VPTS#A and the refer- 
ence time signal exceeds 33 msec, as the threshold 

2t value at Tl. Accordingly, the video reproduction timing 
control means of the video decoder 511 skips one 
frame, which is originally to be reproduced, to adjust the 
reproduction timing so that the difference between the 
VPTS#A and the reference time signal is equal to or 

50 less than the threshold value. 

[0242] The video decoder 521 continues extension 
and reproduction of the compression video signal B, 
and the difference between the VPTS#B and the refer- 
ence time signal exceeds -33 msec, as the threshold 

3£ value at T2. Accordingly, the video reproduction timing 
control means of the video decoder 521 reproduces one 
frame in repetition, which has been already reproduced, 
to adjust the leproduction timing so that the difference 
between the VPTS£B and the reterence time signal is 

<c equal tc or less than the threshold value. 

[0243] Similarly, the video decoder 531 continues 
extension and reproduction of the compression video 
signal C, and the difference between the VPTS#C and 
the reference time signal exceeds 33 msec, as the 

<f threshold value at T3. Accordingly, the video reproduc- 
tion timing control means of the video decoder 531 
skips one frame, which is originally to be reproduced, to 
adjust the reproduction timing so that the difference 
between the VPTS*C and the reference time signal is 

sc equal to or less than the threshold value. 

[0244] As described above, in the third example, 
when the difference between the relerence time signal 
and the VPTS detected by each video decoder exceeds 
the threshold value, the video reproduction timing con- 

£f trol means of each video decoder performs adjustment 
so that difference between the relerence time signal and 
the VPTS does not exceed the threshold value. In this 
manner, the video signals reproduced by vioeodecod- 
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ers can be synchronized with one anothei 
(Example 4) 

[0245] The lourth example lelales to a repioduction 
apparatus toi adjusting a reference time signal using 
audio reproduction time information indicating the time 
to reproduce the audio signal and synchronizes e plural- 
ity ol video signals based on the reference time signal. 
(0246] Figure 41 shows a data structure of an opti- 
cal disk used in an optical disk reproduction apparatus 
in the fourth example. The optical disk includes com- 
pression audio data in addition to the data included in 
the optical disk used in the third example. 
[0247] An audio signal is audio-f i arned in units of 32 
msec, for compression to obtain a compression audio 
stream. The audio stream is packeted in units of 2 kE 
into audio packets and recorded on the optical disk. A 
packet header of each audio packet includes a stream 
ID tor indicating that the stored data is a compression 
audio stream. When the packet stor es a leading pari of 
the audio frame, the packet header alsc includes an 
APTS (audio presentation time stamp) as audio repro- 
duction time information indicating the time to repro- 
duce the frame. 

[0248] Figure 39 is a block diagram of the reproduc- 
tion apparatus in the fourth example. 
[0249] Elements 501 through 532 aie the same as 
those shown in Figure 35 in the third example. 
[0250] Reference numeral 504 represents a buffer 
memory for temporarily storing the compression audio 
signal. Reference numeral 541 represents audio exten- 
sion means for extending the compression audio signal. 
Reference numeral 542 represents a speaker for repro- 
ducing the extended audio signal. 
[0251] Figure 40 shows a structure of the audio 
decoder 541. The audio decoder 541 includes APTS 
detection means 701 tor detecting the APTS stored in a 
packet header of the audio packet, and audio extension 
means 702 for extending the compression audio 
stream. 

[0252] An operation of the optical disk reproduction 
apparatus shown in Figure 39 for reproducing the opti- 
cal disk shown in Figure 41 will be described. 
[0253] The operation until the signal is input to the 
division means 505 is similar to that with the optical disk 
reproduction apparatus in the third example. 
{0254] The data read 1rom the buffer memory 504 is 
divided into compression video signals A through C and 
a compression audio signal by the division means 505 
and output. The division means 505 identifies which of 
the compression video signals A through C and the 
compression audio signal is stored in each packet with 
the stream ID in the packet header of the packeted data, 
and determines the destination based on the identifica- 
tion result. 

[0255] The divided compression video signals and 
compression audio signal are temporarily stored in 



buffer memories 510 through 540 respectively. 
[0256] The video decoders 511 through 531 read 
data from the buffet memories 510 through 530 respec- 
tively, extend the compr ession video signals, and output 

s the signals as video signals to the monitors 51 2 through 
532 respectively. The audio decoder 541 reads data 
from the buffer memory 540. extends the compression 
audio signal, and outputs the signal as an audio signal 
through the speaker 542. 

U) [0257] The operations of the video decoders 511 
through 531 tor extending the compression video sig- 
nals and tor adjusting the synchronization when the dif- 
ference between the reference time signal and the 
VPTS exceeds the threshold value are the same as in 

75 the third example. 

[0258] The compression audio signal read fiom the 
buffer memory 540 is input to the audio decoder 541. 
The APTS detection means 701 delects and outputs the 
APTS. The audio extension means 702 extends the 

20 compression audio stream and outputs the audio signal. 
[0259] The APTS signal output from the audio 
decoder 541 is input to the reference time signal gener- 
ation means 506. and the reference time signal is 
adjusted by the APTS. 

25 [0260] In the fourth example, due to the precision 
error of the crystal oscillator used in the reference time 
signal generation means 506. the video decoders 511 
through 531 and the audio decoder 541. the reference 
time signal is faster in terms of extension and reproduc- 

30 tion relative to the audio decoder 541. The video 
decoder 511 is slower and the video decoder 521 is 
faster in terms of extension and reproduction relative to 
the reference time signal. Unless the reproduction tim- 
ing is adjusted, the reproduced video signals and audio 

35 signal are out of synchronization. 

[0261] Figure 42 is a timing diagram of audio repro- 
duction in the fourth example. Part (a) ol Figure 42 
shows the APTS with respect to reproduction time t. 
Part (b) shows the reference time signal. Part (c) shows 

<o the VPTS*A, at which the compression video signal A 
to be extended by the video decoder 51 1 is to be repro- 
duced, and part (d) shows the VFTSfcB. at which the 
compression video signal B to be extended by the video 
decoder 512 is reproduced. 

<e [0262] Figure 42 does not show the VPTS#C. at 
which the compression video signal C to be extended by 
the video decoder 531 , but the diagram is almost the 
same as in Figure 38 shown regarding the third exam- 
ple. 

so [0263] The reference time signal generation means 
506 is adjusted using the APTS at time when the APTS 
shows tai and ta2. and the reference time signal is 
reset as tai and ta2 at the respective time. 
[0264] The video decoder 51 1 continues extension 

55 and reproduction of the compression video signal A. 
and the difference between the VPTSSA and the refer- 
ence time signal exceeds 33 msec, as the threshold 
value at T4. Accordingly, the video reproduction timing 
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contiol means ol the video decoder 51 1 skips one 
frame, which is originally to be lepioduced. to adjust the 
reproduction timing so that the drffeience between the 
VPTSsA and the reteience time signal is equal to ot 
less than the threshold value 

{0265] The video decodei 521 continues extension 
and lepioduction ot the compression video signal B. 
and the ditleience between the VP! S*6 and the refer- 
ence time signal exceeds -35 msec as the thieshold 
value at T5 and T6. Accordingly, the video lepioduct ion 
timing control means ot the video decodei 521 repro- 
duces one trame in repetition, which has been aheady 
reproduced, to adjust the reproduction timing so that the 
difference between the VPTSfcB and the reterence time 
signal is equal to ot less than the thieshold value. 
(0266] As desaibed above, in the fourth example, 
when the difference between the reference time signal 
and the VPTS detected by each video decodei exceeds 
the threshold value, the video reproduction timing con- 
trol means ot each video decoder performs adjustment 
so that -difference between the reference time signal and 
the VPTS does not exceed the threshold value. In this 
mannei. the video signals reproduced by video decod- 
ers can be synchronized with one another. 
(0267] Regarding the difference between the refer- 
ence time signal and the APTS. the APTS is not 
adjusted using the relerence time signal but the refer- 
ence time signal is adjusted using the APTS. Accord- 
ingly, audio and video signals are synchronized with no 
unnaturalness in the audio output. 

(Example 5) 

{0268] The fifth example relates to a reproduction 
apparatus for adjusting the reference time signal using a 
VPTS detected by one video decoder and synchroniz- 
ing a plurality ol video signals based on the reference 
time signal. 

[0269] Figure 43 is a block diagram of an optical 
disk reproduction apparatus in the trtth example. 
{0270] Elements 501 through 532 are the same as 
those in the third example. Reference numeral 551 rep- 
resents s video decodei used in the fifth example. 
{0271] The video decodei 551 has e function of out- 
putting the detected VPTS. Figure 44 shows a structure 
of the video decoder 551 . 

{0272] The video decoder 551 includes VPTS 
detection means 801 for detecting a VPTS indicating 
the reproduction time ot the video signal multiplexed as 
the compression video signal and video extension 
means 802 for extending the compression video signal. 
{0273] In the fifth example, due to the precision 
error of the crystal oscillator used in the reference time 
signa! generation means 506 and the video decoders 
521 . 531 and 551. the reference time signal is faster in 
terms o; extension and reproduction relative to the 
video decoder 551. The video decoder 521 is slower 
and the video decoder 531 is faster in terms of exten- 



sion and repioduction ieiative to the reference time sig- 
nal. Unless lepioduction timing is adjusted, the 
reproduced video signals are out of synchronization. 
{0274) F igur e 45 is a timing diagram oi video output 

£ in the fifth example Part (a) of Figure 45 shows the 
VPTS*A detected by the video decoder -551 with 
respect to reproduction time t Part (b) shows the reter- 
ence time signal Part (c) shows VPTS#B. at which the 
compression video signal B to be extended by the video 

to decoder 521 is to be reproduced, and part (d) shows the 
VPTS#C. at which the compression video signal C to be 
extended by the video decoder 531 is to be reproduced. 
{0275] The leference time signal generation means 
506 is adjusted using the VPTS#A at time when the 

?£ VPTS#A shows tvi and tv2. and the reference time sig- 
nal is reset as tvT and tv2 at the respective time. 
{0276] The video decoder 521 continues extension 
and repioduction of the compression video signal B. 
and the difference between the VPTS#B and the refer- 

20 ence time signal exceeds 33 msec, as the threshold 
value at T7 Accordingly, the video reproduction timing 
control means of the video decoder 521 skips one 
frame, which is originally to be reproduced, to adjust the 
reproduction timing so that the difference between the 

25 VPTS#B and the reference time signal is equal to or 
less than the threshold value. 

{0277] Similarly, the video decoder 531 continues 
extension and reproduction of the compression video 
signal C, and the difference between the VPTS#C and 

so the reference time signal exceeds -33 msec, as the 
threshold value at T8 and T9. Accordingly, the video 
reproduction timing control means of the video decoder 
531 reproduces one frame in repetition, which has been 
already reproduced, to adjust the reproduction timing so 

3s that the difference between the VPTS#C and the refer- 
ence time signal is equal to or less than the threshold 
value. 

[0278] As described above, in the fifth example, 
when the difference between the reference time signal 

40 and the values ot VPTSs detected by the video decod- 
ers 521 and 531 exceeds the threshold value, the video 
reproduction timing control means of ^ach video 
decodei performs adjustment so that the difference 
between the reterence time signal and the VPTS does 

<5 not exceed the threshold value. 

[0279] By adjusting the reference time signal using 
the VPTS*A detected by the video decoder 551 , the 
video signal reproduced by the video decoder 551 is not 
accompanied by any unnaturalness in the visual output 

56 despite the frame-by-frame skipping or repeat of the 
reproduction Thus, the video signals can be synchro- 
nized with one another. 

(Example 6) 

55 

[0280] The sixth example relates to a reproduction 
apparatus including a plurality of video decoders for 
extending and reproducing a compression video signal. 



26 



51 



EP1 Oil 266 A1 



5? 



Each ol the video decoders includes reieience time sig- 
nal generation means. The reproduction apparatus 
adjusts the ie1eience time signal o1 each video decodei 
using an APTS indicating the time to iepioduce an 
audio signal to lealize synchronization. * 
(0281] In the sixth example, the optical disk shown 
in Figuie 41 is used. 

(0282] Figure 46 is a block diagram ol an optical 
disk leproduction apparatus in the sixth example. 
(0283] Elements 501 through 542 are the same as to 
those shown in Figure 39 in the tourth example. Unlike 
the reproduction apparatus shown in Figure 39. the 
reproduction apparatus in this example does not have 
reieience time signal generation means 506 independ- 
ently, but each video decodei 561 thiough 581 lias ie1- ?5 
eience time signal generation means. 
(0284] Reieience numeral 561 represents a video 
decoder for extending and reproducing compression 
video signal A. reieience numeral 571 represents a 
video decoder 1oi extending and reproducing compres- 20 
sion video signal B, and reieience numeral 581 repre- 
sents a video decodei loi extending and reproducing 
compression video signal C. 

(0285] Figure 47 shows a structure of each ol the 
video decoders 561 through 581 used in the sixth exam- 25 
pie. 

(0286] The video decodei includes VPTS detection 
means 901 lor detecting a VPTS indicating the repro- 
duction time of the video signal multiplexed as the com- 
pression video signal, video extension means 902 lor so 
extending the compression video signal, and video 
reproduction timing control means 903 lor comparing 
the reference time signal and the VPTS and skipping or 
repeating the video reproduction on a freme-by-frame 
basis when the comparison result exceeds the thresh- 55 
old value, and reference time signal generation means 
904 for generating the reference time signal. 
(0287] In the sixth example, the reference time sig- 
nal ol reference time signal generation means 904 . 
included in each of the video decoders 561 through 581 <o 
is adjusted using the APTS detected by the audio 
decoder 541 . 

(0288] Since the reference time signals are 
adjusted using the same APTS. the reieience time sig- 
nals generated in the video decoders 561 through 581 <f 
show the same value after being adjusted. 
[0289] After the adjustment using the APTS, as in 
the fourth example, when the difference between the 
reference time signal and the values of VPTS detected 
by each video decoder exceeds the threshold value, the so 
video reproduction timing control means of each video 
decoder performs adjustment by skipping or repeating 
the reproduction on a frame-by-frame basis so that dif- 
ference between the reference time signal and the 
VPTS does not exceed the threshold value. ££ 
[0290] As described above, in the sixth example, 
the reference time signal generated in each video 
decoder is adjusted using en APTS, and the video 



reproduction timing control means of each video 
decodei maintains the difference between each refer- 
ence time signal and each VPTS to be equal to or less 
than the thieshold value Thus, the video signals can be 
synchronized with one another. 

[0291] As in the tourth example, the audio signal 
and the video signal can be synchronized without pro- 
viding any unnaturalness in the audio output. 
[0292] In the sixth example, the reference time sig- 
nals in the video decoders 561 through 581 are 
adjusted using the APTS detected by the audio decodei 
541 . The video signals can be reproduced in synchroni- 
zation in a similar manner by using one of the video 
decoders shown in Figure 44 in the fifth example and 
adjusting the reference time signals of the other video 
decoders using the VPTS detected by the one video 
decoder. 

(Example 7) 

[0293] The seventh example relates to a reproduc- 
tion apparatus for simultaneously reproducing two com- 
pression video signals. The two compression video 
signals are obtained by dividing a 3D signal into a right- 
eye video signal and a I eft -eye video signal and com- 
pressing the divided video signals. 
[0294] The overall structure of the apparatus is gen- 
erally similar to that ol the optical disk reproduction 
apparatus shown in Figure 46 in the sixth example, but 
the reproduction apparatus in the seventh example 
includes two video decoders for extending -compression 
video signals obtained after the division means 505 
since two video signals are to be reproduced simultane- 
ously. Figure 48 shows a structure of one of the video 
decoders used in the seventh example, and Figure 49 
shows a structure o1 the other video decoder used in the 
seventh example. 

[0295] A£ shown in Figure 48. the video decoder 
includes VPTS detection means 1001 for detecting a 
VPTS indicating the reproduction time of the video sig- 
nal multiplexed as the compression video signal, video 
extension means 1002 for extending the MPEG com- 
piession video signal, reference time signal generation 
means 1004 for generating a reference time signal, and 
video reproduction timing control means 1003 for com- 
paring the reference time signal and the VPTS and skip- 
ping or repeating the video reproduction on a frame-by- 
frame basis when the comparison result exceeds the 
threshold value and also for outputting a horizontal syn- 
chronization signal and a vertical synchronization signal 
for the video to be reproduced. 

[0296] As shown in Figure 49. the other video 
decodei includes VPTS detection means 1101 for 
detecting a VPTS indicating the reproduction time of the 
video signal multiplexed es the compression video sig- 
nal, video extension means 1102 for extending the 
MPEG compression video signal, reference time signal 
generation means 1104 for generating a reference time 
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signal, and video reproduction timing control mean*, 
1103 tot comparing the lefeience time signal and the 
VPTS and skipping or repeating the video reproduction 
on a frame -by -tiame basis when the comparison result 
exceeds the threshold value. ieceiving the horizontal 
synchronization signal and the vertical synchronization 
s*g na: to the video to be reproduced, and also iepio 
duong ne emended vdeo signal in synchronization 
with tne noozoma Jve< nca I synchronization signals 
[0297] The vxiec decoders are connected to each 
othe* so that the nonzomai synchronization signal and 
the vertical syncMionizanon signal output trom the video 
decoder m F»guie 46 a»e sen: to the video decoder in 
Figure 49 

(0298) fcri the outiLdi dn>. tepioduction apparatus in 
the seventh exarnm«- , ^« /,r >j tJHf above-described stiuc- 
tme. the ie*e*enc* b^g^ai operated by each video 
decoder to* the r»gfr o» ey* is edjusaed using an 
APTS. and the vxjoo » ot> ocxjcron few^wog control means 
ot each vrteo oecodo* rn»^tj«-»c 9>e arftc* once between 
each reference time ugrv* and **cf VPTS tc be equal 
to or less than the tr»«*roc %«*^« t>xa th« ught-eye 
vroeo signal and the *«*mc wy^A-' can be syn- 

chronized with one Amf*w* nr « r> fca-ne basis 

By using the hot izom^i m* v^r*rf»nrxzaiion sig- 

nals output by one c* •one ^kxx** me horizon- 
tal and the vertical svncrv-.yvj-a%an worta* o* tne other 
video decoder, two voec vgrw\ ca^ oe reproduced in 
synchronization onap»^t> i> t*as*% 
(0299] In the seveotr r*a^xx :omp»ession video 
signals obtained toon a 21 rt>c ugnai are used and 
div«3ed mto the right -eve eye video signals 

Alternatively, tor exampe a*> o»*^ai wjeo signal hav- 
ing a first resolution is o>v*X2 a vettoca! and/or hori- 
zontal direction into at teas* two w)eo signals including 
a first video signal and a seccrxj *oeo signal having a 
second lesolution which it k**v than the first resolu- 
tion The resunarr signals •** compressed to be used 
Thus, a plurality o* wjec s^gnats r> %/nchr ontzation with 
one another on a p«*e< o> o»e* oavs can be obtained 
as from a 3D video signal B» s^tinesizing such result- 
ant signals, the dear o»grT-§* mo*c signal having the 
original resolution is *v^«> 4 

(Example 8) 

(0300] The eighth ejamp* rotate* to an ootical disk 
reproduction apparatus to* c»ic*x*ng one compression 
video signal and two comp>eu^ort audio signals and 
reproducing the signals simultaneously 
10301] Figure 52 shows a data structure o1 the opti- 
cal disk used in the eigtuh *Ramr*e 
(0302] Twc audio signers D and F are compressed 
to obtain compression audo streams D and E. A video 
signal is compressed to ootam a compression video 
stream. 

{0303] The compression voeo streams D and E 
and the compression vrceo stream are packet ed in units 



oi ? kB into audio packets and video packets A packet 
header oi each packet includes a stream ID tot indicat- 
ing which oi the compression audio sir earns D and E 
and the compression video stream is siored. and the 

* APTS and VPTS described above. 

[0304] Figure 50 is a block diagram o1 a leproduc- 
tion apparatus in the eighth example. 
[0305] The reproduction apparatus has a generally 
similar structure to that in Figure 39 in the fourth exarn- 

io pie. The audio decodei 541 has the same structure as 
that shown in Figure 40, and the video decoder 531 has 
the same structure as that shown in Figure 36 The 
audio decoder 591 has the same structure as that 
shown in Figure 51. 

75 [0306] Relerence numeial 590 represents a buffer 
memory for temporarily storing the compression audio 
signal like the buffer memory 540. Relerence numeral 
592 represents a speaker for reproducing the audio sig- 
nal. 

20 [0307] Figure 51 shows a structure of the audio 
decodei 591. 

[0308] The audio decoder 591 includes APTS 
detection means 1201 for detecting an APTS o! an 
audio signal multiplexed as a compression audio signal. 

25 audio extension means 1202 for extending the com- 
pression audio signal, and audio reproduction timing 
control means 1203 for comparing the reference time 
signal and the APTS and skipping or repeating the 
audio reproduction on an audio frame-by-audio frame 

so basis when the comparison result exceeds the thresh- 
old value. 

[0309] A reproduction operation in the eighth exam- 
ple will be described. 

[0310] The operation until the signal read from the 
5? optical disk 501 is input to the division means 505 is 
similar to that in the other examples. 
[031 1 ] The data read from the buffer memory 504 is 
divided by the division means 505 into a compression 
video signal, the compression audio signal D and the 
to compression audio signal €. and output. The division 
means 505 identifies which of the compression video 
signal, the compression audio signal D and the com- 
pression audio signal E is stored in each packet using 
the packet ID in the packet header of the packet ed date, 
4£ and determines the destination based on the identifica- 
tion result. 

[0312] The divided compression video signal, the 
compression audio signal D and compression audic sig- 
nal E are temporarily stored in buffer memories 530, 

50 540 and 590 respectively. 

[0313] The video decoders reads data from the 
buffer memory 530. extends the compression video sig- 
nal and outputs the signal as a video signal to a monitor 
532. The audio decoders 541 and 591 read data from 

£5 the buffer memories 540 and 590. extend the compres- 
sion audio signals and output the signals as audio sig- 
nals through the speakers 542 and 592. 
[0314] The reference time signal generated by the 
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teleience time signal generation means 506 is adjusted 
by an APTS#D detected by the audio decodei 541 . 
10315] In the audio decodei 591. an APTS#E is 
detected by the APTS detection means 1201 and the 
compiession audio signal E is extended by the audio 
extension means 1202. The audio reproduction timing 
contiol means 1203 receives the extended audio signal 
output from the audio extension means 1202. the refer- 
ence time signal, and the APTS#E tiom the APTS 
detection means 1201. and compares the reference 
time signal and the APTStfE. When the difference 
between the reference time signal and the APTS#E 
exceeds the threshold value, the audio lepioduction 
timing control means 1203 controls the audio r produc- 
tion timing so that the difference is equal to oi less than 
the thi esho*d value 

(0316) In the eighth example. 32 msec is used as 
the thieshorcJ value The audio reproduction timing con- 
trol means 1203 performs the following. 

('«rt^*nc# bn« wgnal-APTS#E)>32 msec.:! audio 

{••«•• «r*# im+ wonal-.APTS#E)<-32 msec.:1 audio 

I03i T) w op^aton of the video decoder 531 for 
eitrxsng jor~x> wucr video signal and performing 
aowvr^r^ r># o>f»e*ence between the reference 

\v~w wtm .^TS exceeds the threshold value 

a»* w^*^- t: ?xm p» Mrd example. 
[03i I) r eiample. due to the precision 

e*'t> & r>» c»-oA-itoi used in the reference time 

s*y>* 3~^-*ion f^m^mt, 506 the video decoder 531. 
anc >caiin 541 and 591 ; the audio decod- 

er i*i »i bCMrcN and the video decoder 531 
is tau#- r* ert^ttjon and reproduction relative 

to '***»«oc # ugnaj Unless reproduction timing 
is ac*AJ*c 'fpoauccc video signals are out of syn- 
ch* Or*.»afron 

|0319] * «^#» S3 a a trrmnQ diagram of video repro- 
ducer r> tr* e.gr*r* c*a~m>e Part (a) of Figure 53 
&mow*> mr ^TTi^o respect to reproduction time t. 
Pan b) ynjvrti ph- • lime signal, part (c) shows 

APTS*£ a* tne uxn> ession audio signal E to be 

e>ienoed b, tr-e auoc decodei 591 is to be reproduced, 
and pan <d> six*** me VP TS at the compression video 
stgnai tc be exi ended b> the video decoder 531 is to be 
reproducec The 'defence tme signal is adjusted using 
the APTS*D when AP7S*D shows ta3 and ta4. The ref- 
erence time signal is reset to ta3 and ta4 at the respec- 
tive time 

[0320] The audio decoder 591 continues extension 
and reproduction of the compression audio signal E. 
and the difference between the AFTS#E and the refer- 
ence time signal exceeds 32 msec, as the threshold 
value at TlO. Accordingly, the audio reproduction timing 
control means 1203 of the audio decoder 591 skips one. 
audio frame, which is originally to be reproduced, to 



adjust the reproduction timing so that the difference 
between the APTS£E and the reference time signal is 
equal to or less than the threshold value. 
[0321] The difference between the VPTS and the 

i reference time signal exceeds -33 msec, as the thresh- 
old value at Tl 1 and T12. Accordingly, the video repro- 
duction timing control means of the video decoder 531 
reproduces one frame in repetition, which has been 
already reproduced at the i espective time, to adjust the 

T6 reproduction timing so that the difference between the 
VPTS and the reference time signal is equal to or less 
than the threshold value. 

(0322] As described above, in the eighth example, 
when the difference between the reference time signal 

75 and the APTS#E detected by the audio decodei 591 
exceeds the threshold value, the audio lepioduction 
timing control means of the audio decoder performs 
adjustment so that difference between the reference 
time signal and the APTS#E does not exceed the 

20 threshold value of audio reproduction. Similarly, differ- 
ence between the reference time signal and the VPTS 
is adjusted so as not to exceed the threshold value of 
video reproduction. In this manner, each audio stgnai 
and the video signal can be synchronized with one 

2S another. 

(Example 9) 

. (0323] In the ninth example, the clock for perform- 
so ing extension is changed for audio reproduction timing 
control. 

[0324] The overall structure and operation of the 
reproduction apparatus in the ninth example are gener- 
ally similar to those of the optical disk reproduction 
5£ apparatus in the eighth example, but the operation of 
audio reproduction timing control performed when the 
reference time signal and the APTS#E exceeds the 
threshold value is different from that of the eighth exam- 
ple. With reference to Figure 54 and 55. audio reproduc- 
er' tion timing control used in the ninth example will be 
described. 

[0325] Figure 54 shows an operation when the dif- 
ference between the APTS#E and the reference timing 
signal exceeds 32 msec., which is the threshold for the 

<i audio reproduction. Part (a) of Figure 54 shows the ref- 
erence time signal with respect to reproduction time t. 
Part (b) shows the APTS#E. and pert (c) shows the 
clock frequency at which the audio decoder 591 per- 
forms extension and reproduction. Ordinary extension 

bo and reproduction are performed by clock fO having a fre- 
quency which is 384 times the sampling frequency fs of 
the audio signal. The difference between the APTS#E 
and the reference time signal exceeds 32 msec, at time 
In, and accordingly, audio reproduction control means 

6< switches the clock fO to fl. The frequency of clock fl is 
higher by "\0% than the frequency of clock fO. Extension 
performed with clock fl proceeds faster than extension 
performed with clock fO by 10%. With clock fl. the 
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extension is performed toi 320 msec ttom the point 
where the difference between the APTS#E and the ref- 
erence lime signal exceeds 32 msec which is the 
threshold value. Thus, the reproduction timing is 
adjusted so that the difference between the APTS#E * 
and the reference time signal is equal to oi less than the 
threshold value. 

(0326] Figure 55 shows an operation when the dif- 
ference between the AFTS#E and the reference timing 
signal exceeds -32 msec, which is the threshold for the io 
audio reproduction. Part (a) ot Figure 55 shows the rel- 
etence time signal with respect to reproduction time t 
Part (b) shows the APTSsE, and pan (c) shows the 
clock frequency at which the audio decoder 591 per- 
forms extension and i epi oduclion. 76 
{0327] The difference between the APTS#E and the 
reference time signal exceeds -32 msec, at time Tl2, 
and accordingly, audic reproduction control means 
switches the clock fO to f2. The frequency of clock f2 is 
lower by 10% than the frequency of clock fO. Extension pc< 
performed with clock t2 proceeds more slowly than 
extension performed with dock fO by 1 0%. With clock f2. 
the -extension is perfor med foi 320 msec, fiom the point 
where the difference between the APTS#E and the ref- 
erence time signal exceeds -32 msec, which is the 2t 
threshold value. Thus, the reproduction timing is 
adjusted so that the difference between the APTS#E 
and the leference time signal is equal to or less than the 
threshold value. 

[0328] As described above, when the difference $c, 
between the APTS#E and the reference time signal 
exceeds the threshold value for the audio reproduction, 
the clock by which the signal is extended is changed so 
that the extension is performed at a higher speed or 
lower speed than the normal speed. By such an opera- 5£ 
tion, the reproduction timing is controlled so that the dif- 
ference between the APTS#E and the reference time 
signal is equal to or less than the threshold value Thus, 
the audio signals and the video signal can be repro- 
duced in synchronization with nc unnaturalness in the <c> 
audio output. 

[0329] In the ninth example, the frequency of the 
clock is changed by 10%. Needless to say, a more nat- 
ural audio signal is obtained by changing the clock less 
or -gradually. 

{0330} In the eighth and ninth examples, the refer- 
ence time signal is adjusted using the APTS#D. Alterna- 
tively, the video decoder shown in Figure 44 can be 
used, in which case the VPTS output from the video 
decoder can be used foi adjustment. sc 
(0331] The present invention has been described 
by way of specific examples. 

[0332] The comparison between the reference time 
signal and the VPTS or AFTS. control of the reproduc- 
tion time, adjustment of the reproduction timing using a £f 
VPTS or APTS can be performed by a microcomputer 
which controls the entirety of the reproduction appara- 
tus. 



1 0333] In the above examples, the present invention 
is applied to optical disk reproduction apparatuses. The 
virtual frame-based editing system according to the 
present invention is also applicable to a reproduction 
apparatus, relened to as the sei top box, tor extending 
compression signals supplied through communication 
networks or digital satellrte broadcasting. Even when 
the piograms are switched, the non-continuous video 
signals are connected seamlessly, which provides a sig- 
nificant advantage. 

INDUS! RIAL APPLICABILITY 

[0334] A basic video signal and an interpolation sig- 
nal are divided into frames each having 1 GOP oi more 
and subjected to interleaving alternately to be recorded 
on the optical disk as interleave blocks. From such an 
optical disk, a high resolution synthesis reproduction 
apparatus reproduces information in both of two types 
of interleave blocks alternately arranged. When the opti- 
cal disk having a high resolution video signal is repro- 
duced by a non-progressive reproduction apparatus, 
information in the interleave block of only odd field or 
even field is reproduced by track jump. Thus, a com- 
plete two-dimensional video is obtained. Thus, compat- 
ibility is realized. 

[0335] Especially, a high resolution video signal 
arrangement information file and a high resolution pic- 
ture identifier are recorded on the optical disk. Accord- 
ingly, the location of the high resolution video signal is 
easily determined. Therefore, two ordinary interlace sig- 
nals can be made into a progressive signal. In addition, 
output ot different content pictures tor the right eye and 
left eye can be avoided. 

[0336] According to the two-stream simultaneous 
reproduction synchronization system according to the 
present invention, an MPEG signal can be virtually 
edited on a trame-by-trame basis, which is convention- 
ally performed only on a GOP-by-GOP basis if deterio- 
ration ot picture Quality should be avoided. By recording 
reproduction control information, signals can be con- 
nected on a trame-by-frame basis when being repro- 
duced. Thus, virtual frame-based editing can be 
realized without deterioration in the picture quality. 
[0337] In the two-stream simultaneous reproduction 
synchronization system, a plurality of <wnpression 
video signals or a plurality of compression audio signals 
can be reproduced in synchronization with one another 
after being extended. 

[0338] In a reproduction apparatus in which the ref- 
erence time signal is adjusted using an APTS detected 
by an audio decoder and the video output timing is con- 
trolled so that the VFTS matches the adjusted reference 
time signal, an audio signal and a plurality of video sig- 
nals are synchronized for reproduction with no unnatu- 
ralness in the audio output. 

[0339] In a reproduction apparatus in which the 
audio output timing is controlled by changing an -exten- 
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sion clock, audio and video signals aie synchronized for 
leproduction with no unnaturalness in the audio output 
with no interruption or pause in the audio signal. 

Claims 

1- An optica! disk reproduction apparatus Ioi repro- 
ducing a signal recorded on an optical disk, in 
which the optical disk has. recorded thereon, at 
least a 1irst video stream representing a low fre- 
quency component ot the video signal and a sec- 
ond video stream representing at least a high 
trequency component o1 the video signal, the first 
video stream includes a plurality o1 tirst interleave 
units and the second video stieam includes a plu- 
rality ol second interleave units, each ol the plurality 
of first interleave units includes ml GOPs (where 
mi is an integer of i or greater), each of the plural- 
ity of second interleave units includes m2 GOPs 
(where m2 is an integer of 1 or greater), the optical 
disk reproduction apparatus comprising: 

a reproduction section for reproducing the first 
video stream and the second video stieam 
recorded on the optical disk; 
a division section for dividing the reproduced 
lirst video stream into the plurality of first inter- 
leave units and for dividing the reproduced sec- 
ond video stream into the plurality ot second 
interleave units; 

a decoding section for decoding the plurality of 
first interleave units to generate a first repro- 
duction signal representing the low frequency 
component of the video signal and for decoding 
the plurality of second interleave units to gener- 
ate a second reproduction signal representing 
at least the high frequency component of the 
video signal; 

a synthesis section tor synthesizing the first 
reproduction signal and the second reproduc- 
tion signal to generate the video signal; and 
an output section tor selectively outputting at 
least one of the first reproduction signal, the 
second reproduction signal, and the video sig- 
nal. 

2. An optical disk reproduction apparatus according to 
claim 1, wherein the plurality of first interleave units 
are each corresponded to lirst time information 
relating to reproduction time, and the plurality of 
second interleave units are each corresponded to 
second time information relating to reproduction 
time. 

3. An optical disk reproduction apparatus according to 
claim 2, further comprising: 

a reference time signal generation section for 



generating a reference time signal: 
a first reproduction control section for control- 
ling the reproduction time of the first reproduc- 
tion signal in accordance with the difference 
5 between the reference time signal and the first 

time information; 

a second reproduction control section for con- 
trolling the reproduction time of the second 
reproduction signal in accordance with the dif- 

io ference between the reference time signal and 

the second time information; and 
an adjusting section tor adjusting the reference 
time signal so that the reference time signal 
supplied to the first reproduction control section 

75 and the i eference signal supplied to the second 

reproduction control section represent sub- 
stantially the same time. 

4. An optical disk reproduction apparatus according to 
20 claim 3. wherein the adjusting section adjusts the 

refer ence time signal based on audio reproduction 
time intor mation representing the time to reproduce 
an audio signal which is to be output in synchroni- 
zation with the video signal. 

25 

5. An optical disk reproduction apparatus according to 
claim 3. wherein, the adjusting section adjusts the 
reference time signal based on at least one of first 
video reproduction time information representing 

30 the time to reproduce the first reproduction signal 
and second video reproduction time information 
representing the time to reproduce the second 
reproduction signal. 

35 6. An optical disk reproduction apparatus according to 
claim S, wherein: 

the lirst reproduction control section controls 
the reproduction time of the first reproduction 
40 signal by skipping a frame of the first reproduc- 

tion signal or by reproducing a frame of the first 
reproduction signal in repetition; and 
the second reproduction control section con- 
trols the repr oduction time of the second repro- 
of duction signal by skipping a frame of the 
second reproduction signal or by reproducing a 
frame of the second reproduction signal in rep- 
etition. 

so 7. An optical disk reproduction apparatus according to 
claim i . wherein at least one of the first time infor- 
mation and the second time information includes at 
least one of a FTS. a DTS and an SCR. 

55 8. An optical disk reproduction apparatus according to 
claim l , wherein: 

the first reproduction signal corresponds to a 
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first pixel numbei. and the second i epi oduction 
signal conesponds to a second pixel numbei 
which is largei than the lirst pixel numbei. 
the synthesis section includes a converter toi 
converting the first reproduction signal into a f 
conversion signal conesponding to the second 
pixel numbei. and 

the video signal is obtained by synthesizing the 
conversion signal and the second lepi oduction 
signal to 

9. An optical disk reproduction apparatus according to 
claim 8. wherein the optical disk furthei has. 
recorded thereon, an identifier representing the first 
p»ei numbei conesponding to the first lepi oduction n 
srgnal. and the conveitei converts the fiist repro- 
duction s«ona! into the conversion signal in accord- 
ance w*th tfie identifier 

10. An optical reproduction apparatus according to 20 

or«<-a' further has. recorded thereon, 
rvntf#t »#<r***s#*rmng the first pixel numbei 
-Dr» MEjnnrrftng m first leproduction signal. 25 

cc*«^» o** » epi oduction apparatus furthei 
r<^K*>r\ * »ot»w control section for control- 
•^c 'oukt & n*e optical disk, and 
r** 'tjtMtxjr xjart?& veoion controls the rotation so 
y v>+ av*~m accordance with the iden- 

11. An or*>c-i ««c>acvction apparatus according to 

Car- ■ m**m+r 55 

n- ort«^: oi> has. recorded thereon. 

a~ Ot*rttrf«* »«c»ea^neing that the video signal 
ootajoec b> ercoong a progressive video 
^«j*>ai o* •»a*T^ t) 30 names per second. <o 
rv» ou£>jr fc^cton ^csuoes a converter for con- 
**-»t»og H-y,^? an* a* the list reproduction sig- 
•v«*i n ^- h*ju**j iiv jduction signal, and the 
wici-o s»gnai into a t»a"ne signal, and 
tne output section outputs the progressive <5 
*«3eo s»gna» ct 00 ^mes per second by output - 
t>ng tne tiame s*gnoi m an overlapping manner 

12. An optical disk reproduction apparatus according to 
claim n. turtnei comprising a buffer memory sec- so 
tion lor storing the plurality of first interleave units 
and the plurality of second interleave units, and the 
buffer memory section has a capacity which is 
equal to or greater than an amount of data of the 
GOP or GOPs included in the second interleave 5£ 
unrts 

1 3. An optical disk reproduction apparatus according to 



claim 12. wherein the buflei memory section has a 
capacity which is 1 MB 01 gieatei 

14. An optical disk including, lecoided thereon, at least 
a first video stream representing a low frequency 
component ol the video signal and a second video 
stream representing at least a high fiequency com- 
ponent of the video signal, wherein: the first video 
stream includes a plurality o1 fust interleave units, 
the second video stream includes a plurality of sec- 
ona interleave units, each ot the plurality of fust 
interleave units includes mi GOPs (where ml is an 
integer ot 1 or greater), and each ot the plurality of 
second interleave units includes m2 GOPs (where 
m2 is an integer of 1 01 gieatei). 

15. An optical disk according to claim 14. wherein the 
plurality of first interleave units and the plurality of 
second interleave units are structured so that repro- 
duction time of one of the plurality of first interleave 
units is substantially equal to reproduction time of 
one of the plurality ot second interleave units, the 
one of the plurality of -second interleave units corre- 
sponding to the one of the plurality of first interleave 
units 

16. An optical disk recording apparatus, comprising: 

a dividing section for dividing a video signal into 
a first video signal representing a low fre- 
quency component of the video signal and a 
second video signal representing at least a 
high frequency component of the video sig- 
nal :an encoding section lor generating a first 
video stream by encoding the first video signal 
and for generating a second video stream by 
encoding the second video signal, wherein: the 
first video stream includes a plurality of first 
interleave units, the second video stream 
includes a plurality of second interleave units, 
each of the plurality of first interleave units 
includes ml GOPs (where mi is an integer of 1 
or greater), and each of the plurality of second 
interleave units includes m2 GOPs (where m2 
is an integer of 1 or greater); 
a selection output section for selectively output- 
ting the plurality of first interleave units included 
in the first video stream and the plurality of sec- 
ond interleave units included in the second 
video stream; and 

a recording section for recording the signal out- 
put from the selection output section on an opti- 
cal disk. 

17. An optical recording apparatus according tc claim 
16. wherein the division section includes a decoder 
for oecoding the first video stream and a differential 
calculator tor calculating a differentia! between the 
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video signal and the signal output tiom the decoder 
and outputs the signal outpul liom the differential 
calculator as the second video signal. 

18. An optical recording apparatus according io claim 6 
17. 

whet ein the division section further includes: 

a first converter for converting the video signal 
into a first conversion signal corresponding to a to 
second pixel numbei which is smaller than a 
first pixel number conespondmg to the v»deo 
signal, and 

a second converter for converting the signal 
output from the decoder into a second convei- is 
sion signal corresponding to the first pixel 
number which is larger than the second pixel 
number corresponding to the signal outpul 
from the decoder. 

wherein, the division section outputs the first 20 
conversion signal as the first video signal, and 
wherein the differential calculator calculates 
the differential between the video signal and 
the second conversion signal. 

19. An optical disk recording apparatus according to 
claim 17. wherein the recording section further 
records on the optical disk an identifier representing 
that the second video signal is outpul from the dif- 
ferential calculator. 30 

20. An optical disk recording apparatus according to 
claim 17, wherein the recording section further 
records on the optical disk an identifier representing 
the first pixel number conesponding to the video 
signal. 

21. An optical disk recording apparatus according to 
claim 17. wherein the recording section furthei 
records on the optical disk an identifier representing <o 
the second pixel number corresponding to the first 
video signal. 

22. An optical disk recording apparatus, comprising . 

At 

an input section for receiving an encoded first 
video stream corresponding to a first pixel 
number and an encoded second video stream 
corresponding to a second pixel number which 
is different from the first pixel number, wherein so 
the first video stream includes a plurality of first 
interleave units, the second video stream 
includes a plurality of second interleave units, 
each of the plurality of first interleave units 
includes ml GOPs (where ml is an integer of 1 
or greater), and each of the plurality of second 
interleave units includes m2 GOPs (where m2 
is an integer of 1 or greater); 



a selection output section for selectively outpul 
ting the plurality of first interleave units included 
in the first video stream and the plurality of sec- 
ond interleave units included in the second 
video stream; and 

a recording section tor recording the signal out- 
put from the selection outpul section on an opti- 
cal disk. 

23. An optical disk reproduction apparatus tor repro- 
ducing a signal recorded on an optical disk, in 
which the optical disk has. recorded thereon, at 
least a first video stream including a plurality of first 
GOPs and a second video stream including a plu- 
rality of second GOPs. each of the plurality of first 
GOPs includes a pluiality of pictures, and each of 
the plurality of second GOPs includes a pluralrty of 
pictures, the optical disk reproduction apparatus 
comprising: 

a reproduction section for reproducing the first 
video stream and the second video stream 
recorded on the optical disk, 
a decoding section for decoding the first video 
stream and the second video stream; and 
an output section for selectively outputting the 
decoded first video stream and the decoded 
second video stream in accordance with repro- 
duction control information, 
wherein the reproduction control information 
indicates that after a first picture included in a 
final first GOP among the pluralrty of first GOPs 
included in the first video stream is reproduced, 
a second picture included in a leading second 
GOP among the plurality of second GOPs 
included in the second video stream is repro- 
duced, the second picture being different from 
a leading picture of the leading second GOP 

24. An optical disk reproduction apparatus according to 
claim 23, wherein the decoding section starts 
decoding the second video stream so that the 
decoding of the second picture has been completed 
when the reproduction of the first picture is com- 
pleted. 

25. An optical disk reproduction apparatus according to 
claim 24, wherein: 

the reproduction control information includes 
information tsl representing a position of the 
first picture, information ts2 representing a 
position of the second picture, and information 
tsG representing a position of the leading pic- 
ture of the leading second GOP, and 
the decoding section finds a decoding st£rt 
position la in accordance with expression 
te=tsv(ts2-tsG) , and starts decoding the sec- 
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ond video stieam based on the decoding slan 
position ta 

26. An optical disk reproduction appaialus according to 
claim 24. wheiem: 

the leproduction control information includes 
timing information representing the timing to 
start decoding the leading second GOP so thai 
reproduction completion time of thelirst picluie 
matches the i epr oduction start time ot the sec- 
ond picture, and 

the decoding section starts decoding the sec- 
ond video stieam based on the timing irrforma- 
tion. 

27. An optical disk reproduction apparatus according to 
claim 23. wheiein the decoding section omits 
decoding ot a picture which is not necessary for 
decoding piclutes tiom the leading picture o1 the 
leading second GOP to the second picture. 

28. An optical disk i epr oduction apparatus according to 
claim 27. wherein the picture which is not neces- 
sary is a B picture 

29. An optical disk leproduction apparatus according to 
claim 23. "further comprising a buffer memoiy sec- 
tion for storing the first video stream and the second 
video stream, and the butler memory section has a 
capacity which is equal to or greater than an 
amount of data of 1 GOP. 

30. An optical disk reproduction apparatus according to 
claim 23, wherein the reproduction section repro- 
duces the reproduction control information 
recorded on the optical disk. 

31. An optical disk reproduction apparatus according to 
claim 23. wherein: 

the optical disk further has, recorded thereon, 
an identifiei representing whethei or not the 
reproduction control information is recordec on 
the optical disk, and 

when the identifier represents that the repro- 
duction control information is recorded on the 
optica! disk, the reproduction section repro- 
duces the reproduction control information 
recorded on the optical disk. 

32. An optical disk reproduction apparatus according to 
claim 23. wherein in a fast reproduction mode, 
when the second picture is not an I picture, the out- 
put section prohibits an ! picture included in the 
leading second GOP from being output 

33. An optical disk reproduction apparatus according to 



claim 32. whei em the output section prohibits a pan 
ot the I pictuie included in the leading second GOP 
from being output based on I picture reproduction 
prohibition mtoimation. 

5 

34. A reproduction control information generation 
apparatus, comprising: 

an input section tor receiving a first video 
to stream including a plurality of first GOPs and a 

second video stieam including a plurality ot 
second GOPs; and 

a generation section for generating reproduc- 
tion control information which represents that 

7£ aftei a first pictuie included in a final first GOP 

among the plurality ot first GOPs included in 
the first video stream is reproduced, a second 
picture included in a leading second GOP 
among the plurality of second GOPs included 

pc in the second video stream is reproduced, the 

second picture being different from a ieading 
picture of the leading second GOP. 

36. A reproduction controt information generation 
25 apparatus according to claim 34. wheiein the repro- 

duction control information includes information 
representing the number of pictures from the lead- 
ing picture of the leading second GOP to the sec- 
ond picture. 

50 

36. A reproduction control information generation 
apparatus according to claim 34, wherein the repro- 
duction control information includes information 
representing the time to reproduce the leading pic- 

3£ ture ol the leading second GOP and the time to 
reproduce the second picture of the leading second 
GOP. 

37. A reproduction control information generation 
<o apparatus according to claim 34, wherein the repro- 
duction control information includes timing informa- 
tion representing the timing to start decoding the 
leading second GOP so that reproduction comple- 
tion time of the first picture matches the reproduc- 

<5 tion start time oi the second picture. 

38. A reproduction control information generation 
apparatus according to claim 37, wherein the timing 
information represents the timing to start decoding 

so the leading second GOP when a picture which is 
not necessary for decoding pictures from the lead- 
ing picture of the leading second GOP to the sec- 
ond picture is not decoded. 

5£ 39. A reproductior control information generation 
apparatus according to claim 38, wherein the pic- 
ture which is not necessary is a B picture. 
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40. An optical disk recording apparatus, comprising: 

a generation section tor generating reproduc- 
tion control information: and 
a tecording section tor recording the reproduc- 
tion control information on an optical disk hav- 
ing, recorded thereon, a liist video stieam 
including a pluiality of lirst GOPs and a second 
video stream including a plurality oi second 
GOPs. 

wherein the reproduction control information 
represents that after a fust picture included in a 
final test GOP among the plurality of first GOPs 
included in the f irst video stream is repioduced. 
a second picture included in a leading second 
GOP among the plurality of second GOPs 
included in the second video stream is repro- 
duced, the second picture being different from 
a leading picture of the leading second GOP. 

41. An optical disk recording apparatus, comprising: 

an editing section for editing a first video 
stream including a plurality of first GOPs and a 
second video stream including a plurality of 
second GOPs so that at least one picture 
unnecessary for reproduction is deleted: and 
a recording section for recording the edited first 
video stream and the edited second video 
stream on an optical disk, 
wherein the reproduction control information 
represents that after a first picture included in a 
final first GOP among the plurality of first GOPs 
included in the first video stream is reproduced, 
a second picture included in a leading second 
GOP among the plurality of second GOPs 
included in the second video stream is repro- 
duced, the second picture being different from 
a leading picture ot the leading second GOP. 

42. An optical recording apparatus according to claim 
41 , wherein the at least one picture unnecessary for 
reproduction includes a picture, of the first video 
stream, after the first picture, and a picture, of the 
second video stream, before the second picture. 



45. An optical recording apparatus according to claim 
41. wheiein the recording section records the 
edited first video stream and the edited second 
video stream in continuous regions of the optical 

5 disk. 

46. An optical recording apparatus according to claim 
4i. wheiein the recoiding section records the 
reproduction control information on the optical disk. 

10 

47. An optical recording apparatus according to daim 
41. wheiein the recording section records the 
leprotiuction control information on a medium other 
than the optical disk. 

15 
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43. An optical recording apparatus according to claim 
42, wherein the at least one picture unnecessary for 
reproduction further includes a picture which is not 
necessary for decoding pictures from the leading so 
picture of the leading second GOP in the second 
video stream until the second picture. 

44. A reproduction control information generation 
apparatus according to claim 43. wherein the at 55 
least one picture unnecessary for reproduction is a 

B picture. 
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